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Introduction 

The  primary  aim  of  this  trial  was  to  evaluate  the  safety  and  feasibility  of  providing  military  service  members  and 
veterans  with  home-  based  telebehavioral  health  (HBTBH)  care  by  comparing  it  to  conventional  in-office  care. 
Depression  is  one  of  the  most  prevalent  psychiatric  conditions  among  Veterans  and  military  personnel  and  it  is 
the  most  frequent  diagnosis  associated  with  psychiatric  hospitalization  in  both  the  active  and  reserve 
components  of  the  U.S.  Armed  Forces  (Anned  Forces  Health  Surveillance  Center,  2013;  Bagalman,  2013). 
Prevalence  rates  for  depression  based  on  screening  data  have  been  estimated  to  be  12%  among  deployed  service 
members  (Gadermann  et  ah,  2012)  and  20%  among  Operation  Iraqi  Freedom/Operation  Enduring  Freedom 
veterans  (Seal  et  ah,  2009;  Corson  et  ah,  2013).  Depression  also  frequently  co-occurs  with  other  conditions, 
such  as  PTSD  and  physical  injuries  among  military  personnel  and  Veterans  and  can  slow  recovery  and  return  to 
duty  among  those  with  comorbid  conditions  (Hoge,  Auchterlonie,  &  Milliken,  2006;  Hoge  et  ah,  2004; 

Milliken,  Auchterlonie,  &  Hoge,  2007).  We  selected  behavioral  activation  (BATD;  Lejuez,  Hopko,  Acierno, 
Daughters,  &  Pagoto,  201 1)  as  the  target  treatment  for  the  present  trial.  Behavioral  activation  treatment  for 
depression  (BATD)  aims  to  reengage  depressed  individuals  in  their  lives  through  focused,  values-based 
activation  strategies.  These  strategies  are  intended  to  counter  patterns  of  negative  affect,  inactivity,  and 
withdrawal  by  reestablishing  naturalistically  reinforcing  behaviors  which  in  turn  alleviate  depressed  mood  and 
create  stable  patterns  of  activity  and  engagement.  BATD  is  also  presumed  to  be  a  compatible  treatment  for 
military  personnel  given  that  it  is  a  problem-focused,  direct  intervention  that  may  be  perceived  as  less 
stigmatizing  than  other  “emotion-focused”  treatments  (Egede,  Frueh,  Richardson,  et  al,  2009). 

A  non-inferiority  design  was  used,  because  it  is  well-suited  for  comparing  novel  adaptations  of  established 
treatments  that  have  demonstrated  efficacy  (Greene,  Morland,  Durkalski,  &  Frueh,  2008).  BATD  has  a  strong 
empirical  evidence-base  (Cuijpers,  van  Straten,  Andersson,  &  van  Oppen,  2008),  and  it  has  been  evaluated  in 
prior  telebhavioral  health  (TBH)  studies  with  veteran  populations  (Egede,  et  al,  2009).  Both  active  duty  service 
members  and  Veterans  were  included  to  provide  preliminary  data  regarding  any  unique  differences  between 
these  populations  and  treatment  settings.  We  hypothesized  that  in-home  care  would  be  no-worse  than  in-office 
care  on  clinical  measures  that  reflect  safety  and  clinical  outcomes  (measures  of  hopelessness  and  depression 
scores)  and  that  participants  in  both  groups  would  show  improved  hopelessness  and  depression  scores  at 
treatment  end.  We  also  predicted  that  BATD  delivered  in  home  would  be  as  safe  as  in-office  treatment.  Our 
assessment  of  safety  additionally  consisted  of  both  qualitative  analysis  of  any  safety  events  requiring  activation 
of  our  safety  protocol  as  well  as  the  evaluation  of  the  relative  clinical  efficacy  of  HBTBH. 

Key  Words 

Behavioral  Activation  (BA)  Cognitive  Behavioral  Therapy  (CBT) 

Depression 

Military 

Post-Traumatic  Stress  Disorder  (PTSD) 

Randomized  Controlled  Trial  (RCT) 

Telehealth 

Body 

Overview  (Pilot) 

A  preliminary  study  was  conducted  to  evaluate  the  feasibility  and  safety  of  providing  U.S.  military  service 
members  with  a  behavioral  health  treatment  delivered  directly  to  the  home  using  videoconferencing.  Ten 
previously  deployed  soldiers  volunteered  to  complete  eight  sessions  of  a  novel  behavioral  activation  treatment 
for  posttraumatic  stress  disorder.  The  primary  clinical  outcomes  assessed  included  symptoms  of  posttraumatic 
stress  and  depression.  Patient  safety  data  and  attitudes  about  seeking  mental  health  services,  treatment 
satisfaction,  treatment  adherence,  and  treatment  compliance  were  also  assessed.  Clinically  significant  reductions 
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in  posttraumatic  stress  symptom  severity  and  depression  symptoms  were  observed.  Soldiers  indicated  high 
levels  of  satisfaction  with  the  treatment,  and  there  were  no  adverse  events  requiring  activation  of  emergency 
safety  procedures.  Technical  problems  associated  with  the  network  were  observed  but  successfully  mitigated. 
The  results  provided  initial  support  for  the  feasibility  and  safety  of  telemental  health  treatments  delivered  by 
videoconferencing  to  the  homes  of  soldiers.  However,  the  optimal  technical  infrastructure  to  support  expansion 
of  synchronous  videoconferencing  capabilities  to  the  home  was  identified  as  an  issue  but  not  resolved.  The 
findings  provide  preliminary  evidence  for  the  feasibility,  safety,  and  high  user  satisfaction  with  home-based 
telemental  health  in  the  military  setting. 

The  objectives  of  the  Pilot  study  were  as  follows: 

1 .  Evaluate  the  safety  of  an  in-home,  web-based  BA  treatment  as  an  intervention  for  PTSD. 

2.  Determine  the  effectiveness  of  in-home  BA  as  treatment  for  PTSD  delivered  via  a  webcam. 

3.  Monitor  PTSD  symptoms  to  detennine  if  there  is  an  association  between  BA  treatment  and  reductions  in 

PTSD  symptomatology. 

4.  Examine  patient  compliance,  treatment  satisfaction,  and  overall  feasibility  of  the  web-based  intervention. 

5.  Identify  best  practices  to  guide  the  implementation  of  other  web-based  tele-mental  health  interventions. 

The  hypotheses  of  the  Pilot  study  were  as  follows: 

1 .  Participants  in  the  in-home  web  care  treatment  condition  will  show  improved  PTSD  scores  at  treatment  end. 

2.  BA  treatment  for  PTSD  delivered  in-home  will  be  safe. 

Participants 

The  sample  consisted  of  10  active  duty  members  of  the  U.S.  Army.  All  participants  were  referred  to  the  study 
from  medical  and  behavioral  health  clinics  at  a  large  Anny  medical  treatment  facility.  The  study  inclusion  and 
exclusion  criteria  were  determined  by  an  initial  screening  interview.  To  be  eligible  for  the  study,  participants 
had  to  endorse  experiencing  at  least  one  criterion  A  stressor  and  have  a  score  of  45  or  higher  on  the  Clinician 
Administered  PTSD  Scale  (CAPS).  The  cutoff  score  of  45  has  been  used  in  other  studies  (Blanchard  et  ah, 

1996;  Weathers,  Huska,  &  Keane,  1991).  Participants  taking  any  psychoactive  medications  had  to  have 
maintained  a  stable  regimen  for  a  minimum  of  30  days  prior  to  study  entry. 

Recruitment 

Enrollment  was  open  to  all  volunteers  who  met  inclusion  criteria.  A  total  of  10  participants  were  recruited  from 
among  all  service  Members  presenting  at-  or  referred  to-  Behavioral  Health  Service  clinics  at  Joint  Base  Lewis- 
McChord.  Participants  included  service  members  from  all  Service  branches;  primarily  Army,  National  Guard, 
and  Reserves. 

Additionally,  participants  that  did  not  meet  full  criteria  for  inclusion  in  the  IRB  approved  study  “Comparing 
Virtual  Reality  Exposure  Therapy  to  Prolonged  Exposure  in  the  Treatment  of  Soldiers  with  PTSD”  (VRET; 
IRBnet  #  1 12226-17,  Gahm  PI)  were  referred  to  this  study  for  assessment.  Participants  in  the  VRET  study  were 
also  selected  from  among  all  Soldiers  presenting  or  referred  to  the  Behavioral  Health  Service  for  treatment  of 
PTSD. 

All  participants  met  diagnostic  criteria  for  PTSD  as  assessed  by  the  CAPS.  The  Diagnostic  and  Statistical 
Manual  of  Mental  Disorders  DSM-IV-TR  research  criteria  for  these  disorders  were  used.  This  study  required  a 
CAPS  score  between  45  and  65  (moderate  PTSD  symptom  severity).  Participants  must  not  have  been 
undergoing  any  other  treatment  for  PTSD. 

Outcome  Measures 

Screening.  Outcomes  were  assessed  at  the  baseline,  mid-treatment,  post- treatment,  and  3 -month  post¬ 
treatment  assessments.  All  assessments  were  video  recorded  and  reviewed  regularly  by  a  supervisory 


psychologist.  The  following  measures  were  assessed: 


Demographic  Questionnaire.  Participants  provided  demographic  infonnation  including  occupation/work 
status/income/living  situation,  branch  of  service/highest  rank,  pain  rating  (0-10),  and  medications. 

Clinician  Administered  PTSD  Scale  (CAPS;  Blake,  et  al.,  1996).  The  CAPS  is  a  structured  interview  that 
assesses  all  DSM-IV  PTSD  criteria  in  terms  of  frequency  and  intensity.  The  CAPS  Current  and  Lifetime 
Version,  which  measures  a  1 -month  symptom  duration,  was  used  for  the  baseline  and  follow-up  assessments. 
The  CAPS  One  Week  Version,  which  measures  symptoms  over  the  past  week,  was  used  to  assess  participants 
after  treatment  sessions  4  and  8.  PTSD  severity,  measured  by  the  CAPS  (total  score),  served  as  the  primary 
PTSD  outcome.  Assessors  were  trained  in  the  administration  of  the  CAPS  and  possessed  prior  experience  with 
this  measure.  Feedback  was  provided  on  an  as  needed  basis  to  improve  compliance  with  CAPS  administration 
rules. 

PTSD  Checklist  Military  Version  (PCL;  Blanchard  et  al.,  1996;  Weathers,  Huska,  &  Keane,  1991).  The 

PTSD  Checklist  17  is  a  self-report  measure  that  evaluates  all  17  DSM-IV  PTSD  symptoms  across  the  three 
primary  symptom  clusters  using  a  5-point  Likert  scale.  Internal  consistency  for  the  total  score  is  high  (0.97),  as 
are  reliability  estimates  (0.96).  The  PTSD  Checklist  Military  Version  (PCL-M)  is  used  here.  A  total  score  of  50 
typically  serves  as  the  threshold  for  identifying  probable  PTSD  among  those  reporting  military-related 
trauma(s). 

Beck  Depression  Inventory-II  (BDI-II;  Beck,  Streer,  &  Brown,  1996).  The  Beck  Depression  Inventory-1118 
(BDI-II)  is  the  most  commonly  used  self-report  measure  of  clinical  depression  severity.  It  consists  of  2 1  items 
that  are  rated  on  a  4-point  scale  and  that  yield  a  range  of  scores  from  0  to  63. 

Beck  Anxiety  Inventory  (BAI;  Beck  &  Streer,  1990).  The  Beck  Anxiety  Inventory  (BAI)  is  a  self-report 
measure  consisting  of  2 1  items  designed  to  discriminate  anxiety  from  depression.  It  has  high  internal 
consistency  (0.92)  and  1-week  test-retest  reliability  (0.75)  and  discriminates  anxious  from  non-anxious 
diagnostic  groups. 

Pittsburgh  Sleep  Quality  Index  (Buysse,  et  al.,  1989).  The  Pittsburgh  Sleep  Quality  Index  (PSQI)  is  a  10-item 
measure  of  sleep  quality.  This  measure  assesses  both  the  quality  and  quantity  of  an  individual’s  sleep  pattern 
over  a  1-month  period.  Internal  consistency  for  this  measure  has  been  found  to  be  0.80,  with  a  reliability 
coefficient  of  0.83  and  test-retest  reliability  of  0.87. 

Safety  measures.  Safety  data  collected  included  any  adverse  events,  psychiatric  hospitalizations,  suicides  and 
nonfatal  suicide-related  behaviors,  number  of  times  the  patient  support  person  was  utilized  during  treatment, 
treatment  adherence,  and  frequency  of  requests  for  patient  or  therapist  technical  support.  Safety-related  data 
were  recorded  after  each  treatment  session  on  the  Treatment  Session  Checklist.  We  also  followed  the  suicide 
assessment  and  risk  management  Standard  Operating  Procedure  (SOP)  used  at  Madigan  Army  Medical  Center 
to  assess  and  document  suicide  risk.  The  SOP  requires  clinicians  to  assess  and  document  current  ideation, 
presence  of  a  plan,  suicidal  intent,  history  of  previous  attempts,  and  degree  of  impulsivity.  Risk  correlates  (e.g., 
recent  loss,  financial  problems),  preparatory  behavior  (e.g.,  available  means),  and  other  risk  factors  (e.g., 
substance  dependence)  are  also  assessed  and  documented.  The  SOP  was  administered  at  the  baseline  assessment 
and  the  first  treatment  session  and  re-administered  at  each  subsequent  session  if  a  patient  endorsed  current 
elevated  risk  per  the  SOP. 

Treatment  Session  Checklist.  The  Treatment  Session  Checklist  is  designed  to  collect  information  for  the 
evaluation  of  clinical  telehealth  sessions.  It  is  used  to  document  safety  information  including  current  suicidal 
ideation,  homicidal  ideation,  and  other  signs  of  risk  (including  the  visual  presence  of  a  weapon  at  the  patient’s 
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location).  Clinical  factors  such  as  indicators  of  intoxication,  disorientation,  and  severe  emotion  dysregulation 
are  also  included,  as  are  questions  related  to  the  in-home  environment,  such  as  “Is  anyone  else  at  home  today?” 
and  “Do  you  feel  that  your  environment  is  safe  and  private?”  This  checklist  was  also  used  to  document 
telehealth  equipment  and  connectivity  status,  adequate  lighting,  and  any  disruptions  to  session. 

Client  Satisfaction  Questionnaire  (CSQ;  Nguyen,  Attkisson,  &  Stegner,  1983).  The  Client  Satisfaction 
Questionnaire  is  an  eight-item  self-report  measure  of  general  satisfaction  with  psychotherapeutic  treatment.  (The 
instrument  is  reproduced  with  permission  of  C.  Clifford  Attkisson.)  Participants  are  asked  to  rate  satisfaction  on 
a  4-point  scale,  with  a  possible  range  of  8  to  32,  with  higher  scores  indicating  greater  satisfaction.  Internal 
consistency  and  construct  validity  have  been  established,  and  the  measure  is  widely  used  in  research. 

Clinical  Outcomes 

Clinically  significant  reductions  in  posttraumatic  stress  symptom  severity  and  depression  symptoms  were 
observed.  Soldiers  indicated  high  levels  of  satisfaction  with  the  treatment,  and  there  were  no  adverse  events 
requiring  activation  of  emergency  safety  procedures.  Technical  problems  associated  with  the  network  were 
observed  but  successfully  mitigated. 


Table  1.  Pilot  study  results. 


MEAN  (SD) 

MEASURE 

BASELINE 

POSTTREATMENT 

t(df) 

HEDGE'S  g 

RC 

CAPS 

82.30  (12.37) 

65.11  (20.74) 

3.29  (8)3 

0.95 

5  improved 

0  deteriorated 

PCL-M 

59.20  (10.40) 

53.22  (14.68) 

2.53  (8)3 

0.44 

7  improved 

1  deteriorated 

BDI-II 

29.50  (10.32) 

20.67  (12.00) 

2.95  (8)3 

0.75 

6  improved 

0  deteriorated 

BAI 

23.20  (11.66) 

19.22  (10.53) 

2.15  (8) 

0.34 

5  improved 

1  deteriorated 

PSQI 

16.20  (3.05) 

15.33  (3.35) 

1.71  (8) 

0.41 

3  improved 

1  deteriorated 

3p<0.05. 

BAI,  Beck  Anxiety  Inventory;  BDI-II,  Beck  Depression  Inventory-ll;  CAPS,  Clinician  Administered  Posttraumatic  Stress 
Disorder  Scale;  PCL-M,  PTSD  Checklist  Military  Version;  PSQI,  Pittsburgh  Sleep  Quality  Index;  RC,  reliable  change;  SD, 
standard  deviation. 

Overview  (RCT) 

The  purpose  of  this  randomized  controlled  non-inferiority  trial  was  to  compare  the  safety,  feasibility,  and 
effectiveness  of  HBTBH  to  care  provided  in  the  traditional  in-office  setting  among  military  personnel  and 
Veterans.  One  hundred  and  twenty  one  U.S.  military  service  members  and  Veterans  were  recruited  at  a  military 
treatment  facility  and  a  Veterans  Health  Administration  (VHA)  hospital.  Participants  were  randomized  to 
receive  eight  sessions  of  behavioral  activation  treatment  for  depression  (BATD)  either  in  the  home  via 
videoconferencing  (VC)  or  in  a  traditional  in-office  (same-room)  setting.  Participants  were  assessed  at  baseline, 
mid-treatment  (4  weeks),  post-treatment  (8  weeks)  and  3  months  post-treatment.  Mixed-effects  modeling  results 
with  Beck  Hopelessness  Scale  (BHS)  and  Beck  Depression  Inventory  II  (BDI-II)  scores  suggested  relatively 
strong  and  similar  reductions  in  hopelessness  and  depressive  symptoms  for  both  groups;  however,  non¬ 
inferiority  analyses  failed  to  reject  the  null  hypothesis  that  in-home  care  was  no  worse  than  in-office  treatment 
based  on  these  measures.  There  were  not  any  differences  found  between  treatment  groups  in  regards  to 
treatment  satisfaction.  Safety  procedures  were  successfully  implemented,  supporting  the  feasibility  of  home- 
based  care.  BATD  was  demonstrated  to  be  feasibly  delivered  to  the  homes  of  active  duty  service  members  and 
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Veterans  via  VC.  Small  group  differences  suggested  a  slight  benefit  of  in-person  care  over  in-home  telehealth 
on  some  clinical  outcomes. 

The  objectives  of  the  RCT  study  were  as  follows: 

1 .  Evaluate  the  safety  of  an  in-home,  web-based  BA  treatment  as  an  intervention  for  depression. 

2.  Detennine  the  effectiveness  of  in-home  BA  as  treatment  for  depression  delivered  via  a  web  cam. 

3.  Monitor  PTSD  symptoms  to  detennine  if  there  is  an  association  between  BA  treatment  and 
reductions  in  PTSD  symptomatology. 

4.  Examine  patient  compliance,  treatment  satisfaction,  and  over  ah  feasibility  of  the  web-based 
intervention  compared  to  in-person  BA  treatment. 

5.  Identify  best  practices  to  guide  the  implementation  of  other  web-based  tele-mental  health 
interventions 

The  hypotheses  of  the  RCT  study  were  as  follows: 

1 .  In-home  care  would  be  no-worse  than  in-office  care  on  clinical  measures  that  reflect  safety  and 
clinical  outcomes. 

2.  Participants  in  both  groups  would  show  improved  hopelessness  and  depression  scores  at 
treatment  end. 

3.  Treatment  delivered  in-home  would  be  as  safe  as  in-office  treatment. 

Participants 

Participants  were  recruited  at  a  large  regional  military  medical  treatment  facility  located  at  Joint  Base  Lewis- 
McChord  (JBLM)  in  Washington  State  and  at  the  VA  Health  Care  System  in  Portland,  Oregon  (VAPHCS). 
Study  patients  at  JBLM  comprised  active  duty,  reserve,  and  National  Guard  service  members.  The  study  patients 
at  the  VAPHCS  site  were  U.S.  military  Veterans  receiving  health  care  services  through  the  VA  hospital. 

Participants  were  included  in  the  study  if  they  met  Diagnostic  and  Statistical  Manual  of  Mental 
Disorders  (4th  ed.,  text  rev.;  DSM-IV-TR;  American  Psychiatric  Association,  2000)  diagnostic  criteria 
for  major  depressive  disorder  or  minor  depressive  disorder  based  on  the  Structured  Clinical  Interview  for 
the  DSM-IV  Axis  I  Disorders,  Research  Version,  Patient  Edition  (SCID-EP;  First,  Spitzer,  Gibbon,  & 

Williams,  2002).  The  inclusion  of  both  major  and  minor  depressive  disorder  in  the  trial  provided  for  a 
more  generalizable  representation  of  patients  typically  seen  at  the  sites.  Clinical  assessors  were  unaware 
of  treatment  condition  throughout  the  trial.  All  participants  had  to  have  access  to  high  speed  Internet 
available  in  their  homes. 

Recruitment 

Recruitment  occurred  primarily  through  referral  from  medical  and  behavioral  health  clinical  providers  at 
the  study  sites  who  were  educated  about  the  inclusion/exclusion  criteria  of  the  study.  This  occurred  from 
a  variety  of  clinics  including  outpatient  behavioral  health  offices,  Soldier  Readiness  Centers,  Post¬ 
deployment  Health  and  Readiness  Assessment  offices,  Family  Medicine  Clinics,  and  Ministry  and 
Pastoral  Care  offices. 

Potential  participants  could  also  self-select  to  contact  the  study  for  consideration.  Study  infonnation  was 
posted  on  flyers  at  the  medical  centers  at  both  sites.  At  JBLM,  study  infonnation  was  also  available  on 
large  banners  posted  at  the  base  exit  points.  Information  was  also  available  online  through  a  webpage 
created  at  the  National  Center  for  Telehealth  and  Technology  (T2),  which  could  be  disseminated  via 
T2’s  extant  social  media  outlets. 

Outcome  Measures 


Screening 
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Prior  to  randomization,  patients  were  assessed  on  the  following  inclusionary  and  exclusionary  criteria: 


a.  Inclusion  Criteria  - 

(a)  Current  Major  Depressive  Disorder  or  Minor  Depressive  Disorder 

(b)  High-speed  internet/network  access  at  home  (384kbs  minimum) 

(c)  Informed  consent 

(d)  Fluent  in  the  English  language 

b.  Exclusion  Criteria  - 

(a)  Currently  undergoing  psychotherapy  for  depression 

(b)  <18  or  >65  years  of  age 

(c)  Active  psychotic  symptoms/disorder  as  determined  by  the  SCID  for  DSM-IV 

(d)  Dysthymic  Disorder 

(e)  Current  suicidal  ideation  with  intent  or  recent  (within  six  months)  history  of  a  suicide  attempt 

(f)  History  of  Organic  Mental  Disorder 

(g)  Current  substance  dependence  as  detennined  by  the  SCID  (lifetime  substance  dependence  or  substance  abuse 
will  not  be  excluded) 

(h)  History  of  violence  or  poor  impulse  control  causing  potential  risk  to  study  staff  or  others 

(i)  Significant  ongoing  stressors  that  require  urgent  crisis  intervention 

(j)  Have  a  living  arrangement  that  will  not  permit  the  use  of  a  private  space  to  participate  in  the  study 

Clinical  Outcomes 

Beck  Hopelessness  Scale  (BHS;  Beck,  Weissman,  Lester,  &  Trexler,  1974):  The  BHS  comprises  20  true- 
false  statements  relating  to  feelings  of  hopelessness  about  the  future.  After  reverse  scoring  several  items,  sum 
scores  are  calculated  with  a  possible  range  of  0  to  20,  with  higher  scores  indicating  higher  levels  of 
hopelessness. 

Beck  Depression  Inventory  (BDI-II;  Beck,  Streer,  &  Brown,  1996):  The  BDI-II  comprises  21  items  with 
four  response  categories  for  each  item.  Sum  scores  are  calculated  with  a  possible  range  of  0  to  63,  with  higher 
scores  indicating  higher  levels  of  depression  symptom  severity. 

Structured  Clinical  Interview  for  DSM-IV  (SCID-I/P;  First  et  al.,  2002):  The  SCID-I/P  was  used  for  initial 
diagnostic/screening  purposes  and  at  follow-up  assessments  for  the  presence  of  Major  Depressive  Disorder 
(MDD).  The  SCID-EP  has  excellent  established  inter-rater  reliability  (overall  kappa  =  0.85). 

Beck  Anxiety  Inventory  (BAI;  Beck  &  Streer,  1990):  The  BAI  is  a  self-report  measure  consisting  of  21  items 
designed  to  discriminate  anxiety  from  depression.  Similar  to  the  BDI-II,  the  sum  scores  range  from  0  to  63. 

PTSD  Checklist  -  Military  Version  (PCL-M;  Weathers,  Huska,  &  Keane,  1991) 

PTSD  Checklist  Military  Version  (PCL;  Blanchard  et  al.,  1996;  Weathers;  Huska,  &  Keane,  1991):  The 

PCL-M  is  a  self-report  measure  that  evaluates  all  17  DSM-IV-TR  PTSD  symptoms  across  the  three  primary 
symptom  clusters  using  a  5-point  Likert-type  scale.  A  total  score  of  50  typically  serves  as  the  threshold  for 
identifying  probable  PTSD  among  those  reporting  military  related  trauma(s). 

Inventory  of  Attitudes  Toward  Seeking  Mental  Health  Services  (IASMHS;  Mackenzie,  Knox,  Gekoski,  & 
Macaulay,  2004):  The  IASMHS  is  a  24-item  assessment  of  help-seeking  attitudes.  It  includes  the  following 
three  factors  based  on  components  of  Ajzen’s  Theory  of  Planned  Behavior  (Ajzen,  1985):  psychological 
openness,  help-seeking  propensity,  and  indifference  to  stigma.  Test-retest  reliability  for  the  factors  ranges  from 
moderate  to  high.  Convergent  validity  has  been  demonstrated  by  effectively  differentiating  those  who  would  and 
would  not  use  services.  The  IASMHS  was  completed  at  baseline,  post  treatment,  and  follow-up. 
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Client  Satisfaction  Questionnaire  (CSQ;  Nguyen,  Attkisson,  &  Stegner,  1983)  :  The  CSQ-8  is  an  8-item 
self-report  measure  of  general  satisfaction  with  psychotherapeutic  treatment.  Participants  are  asked  to  rate 
satisfaction  on  a  4-point  scale,  with  a  possible  range  of  8  to  32,  with  higher  scores  indicating  greater 
satisfaction.  Internal  consistency  and  construct  validity  have  been  established  and  the  measure  is  widely  used  in 
research.  The  CSQ  was  administered  at  post  treatment. 

Treatment  Session  Checklist  (Luxton  et  al.  2014):  Safety  related  data  were  recorded  after  each  treatment 
session  on  the  treatment  session  checklist  (see  Luxton  et  al.  2014).  This  checklist  is  designed  to  collect 
infonnation  for  the  evaluation  of  clinical  telehealth  sessions.  It  is  used  to  document  pertinent  safety  information 
including  current  suicidal  ideation,  homicidal  ideation,  the  presence  of  a  firearm  at  the  patient’s  location,  and 
signs  of  intoxication,  disorientation,  and  severe  emotion  dysregulation.  There  are  also  questions  related  to  the 
in-home  environment,  such  as,  “Is  anyone  else  at  home  today?”  and  “Do  you  feel  that  your  environment  is  safe 
and  private?”  This  checklist  is  also  used  to  document  telehealth  equipment  and  network  connectivity  status, 
lighting,  and  any  disruptions  to  assess  technical  feasibility. 

Demographic  questionnaire:  Participants  provided  demographic  information  including  occupation/work 
status/income/living  situation,  branch  of  service/highest  rank,  pain  rating  (0-10),  and  medications. 

Results 

A  total  of  40  participants  completed  all  eight  sessions  in  the  in-home  condition,  and  42  completed  all  eight 
sessions  in  the  in-office  condition  for  a  total  attrition  rate  of  32.23%.  In  the  in-home  condition,  there  were  16 
withdrawals,  three  losses  to  follow-up,  and  three  who  did  not  begin  treatment.  In  the  in-person  condition,  there 
were  nine  withdrawals,  five  losses  to  follow-up,  and  three  who  did  not  begin  treatment.  The  difference  in 
proportions  of  subjects  that  did  not  complete  treatment  between  the  groups  was  not  statistically  significant  (in- 
home  =  35.48%,  in-person  =  28.81%,  %2  =  0.62,  df  =  1  ,p  =  .433).  Baseline  scores  on  the  BHS  and  the  BDI-II 
were  not  associated  with  dropout.  None  of  the  demographic  variables  were  associated  with  dropout  except  race; 
however,  the  category  that  showed  the  strongest  association  with  dropout  was  the  “other”  category  which 
included  only  ten  participants.  This  evidence  suggested  that  missing  at  random  was  a  reasonable  assumption  for 
analysis. 

Primary  outcomes 

At  post  treatment,  participants  in  the  in-person  group  had  an  average  reduction  of  6.21  points  on  the  BHS  (95% 
Cl  =  -7.38,  -5.05)  and  17.63  points  on  the  BDI-II  (95%  Cl  =  -20.21,  -15.06).  Participants  in  the  in-home  group 
had  an  average  reduction  of  3.91  points  on  the  BHS  (95%  Cl  =  -5.25,  -2.57)  and  13.40  points  on  the  BDI-II 
(95%  Cl  =  -16.36,  -10.44).  For  both  outcomes,  the  magnitude  of  decrease  over  time  was  less  pronounced  for  the 
in-home  group  compared  to  the  in-person  group.  After  standardization  of  the  difference  between  the  groups  at 
post-assessment,  we  found  that  the  upper  bound  of  the  90%  confidence  interval  included  0.50.  The  per-protocol 
analysis  provided  similar  results;  however,  the  point  estimates  were  somewhat  greater  in  magnitude  than  those 
from  the  intent-to-treat  analysis.  See  Figures  3  and  4  for  a  graphic  display  of  the  point  estimates  and  confidence 
intervals  against  the  non-inferiority  margin. 

Secondary  outcomes 

At  baseline,  56  participants  (90.32%)  in  the  in-home  group  and  54  in  the  in-person  group  (91.53%)  met  SCID- 
FP  criteria  for  major  depressive  disorder.  At  post  treatment,  there  were  8  participants  in  the  in-home  group 
(17.78%)  and  6  in  the  in-person  group  (14.29%)  who  met  criteria  for  major  depressive  disorder.  The  difference 
in  reduction  of  the  number  of  participants  meeting  criteria  for  major  depressive  disorder  was  not  statistically 
significant  (b  =  0.33,  95%  Cl  =  -1.22,  1.88).  By  the  twelve-week  follow-up,  4  participants  in  the  in-home  group 
(9.52%)  and  8  in  the  in-person  group  (22.22%)  met  criteria  for  major  Depressive  Disorder.  This  difference  was 
also  not  statistically  significant  (b  =  -0.89,  95%  Cl  =  -2.54,  0.77).  Participants  in  both  treatment  groups  reported 


reductions  in  anxiety  and  posttraumatic  stress  symptoms  and  in  mental  health  treatment  stigma  as  measured  by 
the  IASMHS.  There  were  no  statistically  significant  differences  between  the  treatment  groups  on  these 
outcomes.  Average  scores  on  the  CSQ  suggested  a  high  level  of  treatment  satisfaction  for  both  treatment  groups. 
There  was  no  statistically  significant  difference  between  the  treatment  groups  on  the  CSQ. 

Post  hoc  analysis 

We  did  not  identify  a  large  change  in  the  magnitude  of  the  differences  between  the  treatment  groups  when 
including  demographic  covariates  in  the  model  or  when  restricting  the  analytic  sample  to  those  at  the  military 
treatment  facility  (MTF)  site  and  those  with  an  initial  major  depressive  disorder  diagnosis.  This  suggested  that 
the  conclusions  from  the  primary  models  were  robust  to  the  heterogeneity  introduced  by  including  veterans  and 
participants  with  minor  depression  into  the  study  sample.  However,  given  the  small  number  of  veterans  and 
participants  in  the  sample,  we  were  not  able  to  formally  test  for  effect  measure  modification  in  the  association 
between  treatment  and  the  primary  and  secondary  outcomes. 

Using  the  95%  confidence  interval  (Cl)  as  opposed  to  the  90%  confidence  interval  allowed  for  a  two-tailed  test 
of  non-inferiority  and  inferiority.  The  95%  Cl  for  the  unstandardized  difference  on  the  BHS  ranged  from  0.33  to 
4.20  for  the  intent-to-treat  analysis  and  from  0.57  to  4.67  for  the  per  protocol  analysis,  both  of  which  suggested 
that  the  in-home  method  of  delivery  was  inferior  to  the  in-person  method  of  delivery.  The  post  hoc  power  for 
the  intent-to-treat  analysis  was  0.81.  The  95%  confidence  intervals  for  the  unstandardized  differences  on  the 
BDI  ranged  from  -0.03  to  8.50  and  from  0.40  to  9.46  for  the  intent-to-treat  and  per  protocol  analyses, 
respectively.  Since  the  confidence  interval  for  the  intent-to-treat  analysis  covered  0,  the  results  were 
inconclusive  as  to  inferiority  of  the  in-home  method  of  delivery  on  treatment  efficacy.  The  post  hoc  power  for 
the  intent-to-treat  analysis  of  the  BDI-II  was  0.69. 

Dropout  by  the  mid-  and  post-treatment  assessments  was  not  associated  with  treatment  assignment,  prior 
outcome  scores,  or  the  current  outcome  scores.  Estimates  of  the  treatment  differences  at  mid  and  post  treatment 
for  the  BHS  and  the  BDI-II  were  similar  to  those  presented  in  Table  3  for  the  intent-to-treat  models.  These 
findings  further  supported  an  assumption  of  data  at  least  missing  at  random. 

There  were  seven  participants  who  had  adverse  events  that  required  reporting  in  the  in-home  group  and  four  in 
the  in-person  group.  None  of  these  adverse  events  (e.g.  a  severe  exacerbation  of  asthma  symptoms)  were 
determined  to  be  related  to  study  procedures.  The  safety  protocol  was  initiated  one  time;  this  occurred  for  a 
military  service  member  in  the  in-home  condition  who  contacted  his  study  provider  and  presented  in-person  to 
the  research  staff.  The  participant  reported  that  he  was  experiencing  distress  and  had  underreported  baseline 
level  of  suicidal  ideation  during  the  intake  assessment.  The  patient  was  assessed  by  a  supervisory  psychologist 
and  escorted  to  the  emergency  department  for  further  evaluation  as  per  the  established  safety  protocol. 
Additional  description  of  suicide  risk  management  used  in  the  trial  is  provided  by  Luxton  et  al.  (2014). 

Of  the  378  treatment  sessions  completed  in  the  in-home  group,  there  were  190  (50.26%)  VC 
connectivity  issues  including  the  inability  to  initiate  a  webcam  connection  (n  =  137,  36.34%)  and  the 
inability  to  maintain  a  webcam  connection  once  initiated  (n  =  66,  20.31%).  Overall,  135  (35.71%) 
treatment  sessions  required  a  phone  call  to  resolve  a  technical  issue  or  to  complete  all  or  a  portion  of  the 
session’s  activities  because  of  technical  problems. 

To  improve  the  applicability  of  these  findings  to  the  Military  Health  System,  and  economic  analysis  of  both 
delivery  modalities  was  conducted.  This  analysis  found  that  the  total  direct  cost  of  HBTBH  care  was  higher 
than  IP  care,  but  this  was  sensitive  to  whether  patients  already  have  access  to  computer  technology  suitable  for 
video  conferencing.  Patients  with  access  to  computer  technology  may  accrue  less  costs  while  maintaining 
quality  health  care  for  depression.  However,  if  a  large  proportion  of  patients  are  without  the  necessary  computer 
technology,  the  total  direct  costs  of  the  HBTBH  is  largely  driven  by  the  need  for  government-supplied  computer 


technology.  Health  care  policies  centered  on  implementation  of  HBTBH  care  may  benefit  from  an  initial 
assessment  of  A/V  coverage  among  beneficiaries. 
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Table  2.  Intent-to-treat  and  per  protocol  analyses  and  internal  consistency  reliability  estimates  of  the  BHS  and  BDI-II  against  a  non-inferiority  margin  of  a 
standardized  difference  of  0.50. 


In-home 

In-Person 

Outcome  and 
time 

n 

M  (SD) 

a 

n 

M  (SD) 

a 

b 

90%  Cl 

B 

90%  Cl 

Intent-to-treat 

BHS 

Pre 

62 

9.00  (5.12) 

0.88 

59 

10.37  (6.13) 

0.92 

Ref. 

Ref. 

Mid 

48 

7.21  (5.66) 

0.88 

44 

7.95  (6.26) 

0.94 

1.29 

-0.30,2.88 

0.22 

-0.05,0.50 

Post 

45 

4.89  (4.64) 

0.90 

42 

4.43  (4.94) 

0.92 

2.30 

0.68,  3.92 

0.40 

0.12,0.68 

Follow-up 

42 

5.21  (5.10) 

0.92 

36 

5.53  (5.97) 

0.94 

1.63 

-0.05,  3.32 

0.28 

-0.01,0.58 

os  =  5.77 
ICC  =  0.64 

BDI 

Pre 

62 

27.60  (10.45) 

0.88 

59 

29.71  (11.33) 

0.89 

Ref. 

Ref. 

Mid 

47 

19.40  (11.77) 

0.93 

44 

20.20  (13.09) 

0.93 

1.70 

-1.81,5.22 

0.14 

-0.15,0.44 

Post 

45 

13.82  (12.02) 

0.94 

42 

11.74  (12.08) 

0.95 

4.23 

0.66,  7.81 

0.36 

0.06,0.66 

Follow-up 

42 

14.76  (12.89) 

0.95 

36 

15.00  (12.61) 

0.93 

1.89 

-1.84,  5.61 

0.16 

-0.16,0.48 

os  =  11.78 
ICC  =  0.60  _ 

Per  protocol 

BHS 

Pre 

40 

8.58  (5.38) 

0.90 

42 

11.00  (6.27) 

0.93 

Ref. 

Ref. 

Mid 

40 

7.28  (5.67) 

0.87 

42 

7.55  (6.09) 

0.93 

2.15 

0.43,  3.87 

0.39 

0.08,0.70 

Post 

40 

4.63  (4.42) 

0.89 

42 

4.43  (4.94) 

0.92 

2.62 

0.90,  0.43 

0.47 

0.16,0.78 

Follow-up 

38 

4.82  (4.99) 

0.92 

36 

5.53  (5.97) 

0.94 

1.90 

0.12,  3.68 

0.34 

0.02,  0.67 

os  =  5.53 
ICC  =  0.63 

BDI 

Pre 

40 

26.65  (11.80) 

0.90 

42 

29.69  (11.74) 

0.90 

Ref. 

Ref. 

Mid 

39 

18.97  (11.89) 

0.93 

42 

19.12  (12.04) 

0.92 

2.74 

-1.08,  6.56 

0.22 

-0.09,0.54 

Post 

40 

13.63  (12.47) 

0.95 

42 

11.74  (12.08) 

0.95 

4.93 

1.12,  8.73 

0.40 

0.09,0.71 

Follow-up 

38 

14.58  (13.18) 

0.95 

36 

15.00  (12.61) 

0.93 

2.64 

-1.29,  6.58 

0.22 

-0.11,0.54 

Os  =  12.23 
ICC  =  0.63 

Note:  BHS  =  Beck  Hopelessness  Scale;  BDI-II  =  Beck  Depression  Inventory  —  II;  b  =  unstandardized  difference  between  in-home  and  in-person  treatment  groups;  Cl  =  confidence 
interval;  B  =  standardized  difference  between  in-home  and  in-person  treatment  groups  using  the  baseline  standard  deviation;  ICC  =  intraclass  correlation  (time  nested  within  subject). 
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Table  3.  Intent-to-treat  analysis  and  internal  consistency  reliability  estimates  of  secondary  outcomes. 


In-home 

In-person 

Outcome  and 
time 

n 

M  (SD) 

a 

n 

M  (SD) 

a 

b 

95%  Cl 

B 

BAI 

Pre 

62 

15.48  (10.40) 

0.90 

59 

16.85  (12.92) 

0.94 

Ref. 

Ref. 

Mid 

48 

12.21  (9.75) 

0.91 

44 

13.00  (10.98) 

0.93 

1.29 

-2.23,  4.82 

0.12 

Post 

45 

9.71  (8.67) 

0.90 

42 

8.31  (9.11) 

0.92 

3.13 

-0.46,  6.74 

0.30 

Follow-up 

42 

11.10(8.63) 

0.88 

36 

9.75  (8.95) 

0.92 

3.10 

-0.64,  6.84 

0.29 

(7S  =  10.56 
ICC  =  0.65 
PCL-M 

Pre 

62 

43.15  (13.53) 

0.91 

59 

45.17  (14.75) 

0.92 

Ref. 

Ref. 

Mid 

48 

37.60  (13.82) 

0.92 

44 

37.41  (16.24) 

0.95 

2.20 

-2.18,  6.58 

0.15 

Post 

45 

33.16  (14.69) 

0.94 

42 

32.43  (16.58) 

0.96 

2.14 

-2.33,  6.61 

0.15 

Follow-up 

42 

35.05  (14.57) 

0.93 

36 

34.39  (14.72) 

0.93 

2.26 

-2.39,  6.91 

0.15 

(Ts=  14.67 
ICC  =  0.72 
IASMHS 

Pre 

62 

35.23  (14.40) 

0.83 

59 

31.47  (14.33) 

0.83 

Ref. 

Ref. 

Post 

45 

27.07  (14.98) 

0.87 

42 

21.88  (13.75) 

0.85 

1.53 

-3.46,  6.52 

0.10 

Follow-up 

42 

28.33  (17.31) 

0.91 

36 

23.69  (16.57) 

0.90 

0.24 

-4.97,  5.44 

0.02 

(7s  =  15.16 
ICC  =  0.68 
CSQ 
Post 

45 

28.76  (3.41) 

0.91 

42 

29.29  (3.98) 

0.92 

-0.53 

-2.11,-1.05 

-0.14 

Note :  b  =  unstandardized  difference  between  in-home  and  in-person  treatment  groups;  Cl  =  confidence  interval;  B  =  standardized  difference  between  in-home  and  in-person  treatment 
groups  using  the  baseline  standard  deviation;  ICC  =  intraclass  correlation  (time  nested  within  subject);  BAI  =  Beck  Anxiety  Inventory;  PCL-M  =  PTSD  Checklist  -  Military;  IASMHS 
=  Inventory  of  Attitudes  toward  Seeking  Mental  Health  Service. 
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Challenges 

A  summary  of  challenges  throughout  this  project  has  been  compiled  for  this  report. 

Year  1:  The  IRB  and  HRPO  approval  process  was  a  major  challenge  due  to  frequent  delays  in  the 
review/approval  process  that  were  outside  the  control  of  study  staff. 

Year  2:  Madigan  IRB  and  HRPO  approval  continued  being  a  consistent  challenge  during  year  2,  namely 
because  of  delays  in  the  review/approval  process  for  both  sites.  However,  approvals  were  obtained,  and  both 
sites  began  enrolling  participants. 

Additionally,  referral-based  recruitment  strategies  also  proved  challenging.  However,  the  early  phase  of  the 
recruitment  period  was  spent  developing  relationships  with  providers  who  were  willing  to  refer  participants,  and 
enrollment  steadily  increased  to  the  point  where  our  quarterly  recruitment  goals  were  being  met  consistently. 

Year  3:  Network  security  changes  made  by  the  Anny  resulted  in  a  2.5-month  period  from  July- 
September  2013  during  which  participants’  attempts  to  access  the  Jabber  video-conferencing  software 
(In-Home  Treatment  condition)  were  blocked  by  the  Army  network.  We  were  able  to  partially  resolve 
this  issue,  but  it  did  present  a  major  challenge  that  interrupted  the  delivery  of  the  treatment  protocol 
during  that  time  frame. 

Year  4:  An  important  use  of  the  data  generated  by  this  trial  was  the  examination  of  economic 
costs/savings  of  home-based  care.  Finding  the  right  group  of  health  economic  experts  to  conduct  the 
appropriate  analyses  was  challenging,  but  we  were  able  to  identify  a  productive  and  skilled  research 
group  at  the  University  of  Washington’s  Department  of  Pharmacy  who  were  capable  of  completing  this 
sophisticated  analysis. 

Key  Research  Accomplishments 

•  Main  findings  from  this  project  were  presented  to  leadership  of  the  Military  Health  System 
(MHS)  at  the  request  for  guidance  on  the  use  of  Telehealth  in  the  MHS. 

•  Main  findings  are  under  consideration  for  publication  in  the  Journal  of  Consulting  and  Clinical 
Psychology. 

•  Findings  from  the  Pilot  Study  have  been  published  in  the  journal  Telemedicine  and  eHealth. 

•  Main  findings  presented  at  the  Military  Health  System  Research  Symposium  (MHSRS)  as  a 
Plenary  Presentation. 

•  Study  staff  was  awarded  a  2015  MHSRS  Research  Team  Award. 

•  6  other  peer-reviewed  publications  and  one  book  chapter  have  been  published  regarding  this 
study. 

•  Results  from  the  economic  analysis  will  be  presented  at  the  International  Society  for 
Phannacoeconomics  and  Outcomes  Research  annual  conference. 

•  4  additional  publications  are  in  development. 

Reportable  Outcomes 

Peer  Reviewed  Publications: 

Luxton,  D.D.,  Pruitt,  L.D.,  Wagner,  A.,  Smolenski,  D.J.,  Jenkins-Guarnieri,  M.A.,  &  Gahm,  G.  (In  Submission). 
Home-based  telebehavioral  health  for  U.S.  military  personnel.  Journal  of  Consulting  and  Clinical 
Psychology. 
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Luxton,  D.D.,  Pruitt,  L.D.,  O’Brien,  K.,  &  Kramer,  G.  (2015).  An  evaluation  of  the  feasibility  and  safety  of  a 
home-based  telemental  health  treatment  for  PTSD  in  the  US  military.  Telemedicine  Journal  and 
eHealth,  21,  1-7.  doi:  10.1089/tmj.2014.0235 

Jenkins-Guamieri,  M.,  Pruitt,  L.D.,  Luxton,  D.D.,  Johnson,  K.  (2015).  Patient  perceptions  of  telemental  health: 
Systematic  review  of  direct  comparisons  to  in-person  psychotherapeutic  treatments.  Telemedicine 
Journal  and  eHealth,  21,  8.  doi:  10.1089/tmj.2014.0165 
Luxton,  D.D.,  O’Brien,  K.,  Pruitt,  L.D.,  Kramer,  G.,  Johnson,  K.  (2014).  Managing  suicide  risk  in  home-based 
telepractice.  International  Journal  of  Psychiatry  in  Medicine,  48,  19-31.  doi: 
http://dx.doi.org/10.2190/PM.48.Lc 

Luxton,  D.D.,  Pruitt,  L.D.,  O’Brien,  K.,  Stanfill,  K.,  Jenkins-Guamieri,  M.A.,  Johnson,  K.,  et  al.  (2014).  Design 
and  methodology  of  a  randomized  clinical  trial  of  home  based  telemental  health  treatment  for  U.S. 
military  personnel  and  veterans  with  depression.  Contemporary  Clinical  Trials,  38,  134-144.  doi: 
10.1016/j.cct.2014.04.002 

Luxton,  D.D.,  Pruitt,  L.D.,  &  Osenbach,  J.E.,  (2014).  Best  practices  for  remote  psychological  assessment  via 
telehealth  technologies.  Professional  Psychology:  Research  and  Practice,  45,  27-35.  doi: 
10.1037/a0034547. 

Pruitt,  L.D.,  Luxton,  D.D.,  Shore,  P.  (2014).  Additional  clinical  benefits  of  home-based  telemental  health 
treatments.  Professional  Psychology:  Research  and  Practice,  45,  340-346.  doi:  10.1037/a0035461 

Solicited  Book  Chapters: 

Luxton,  D.,  Pruitt,  L.D.,  &  Jenkins,  M.  (2015).  Telebehavioral  assessment.  In  Teurk  &  Shore  (Eds.),  Clinical 
Videoconferencing  in  Telehealth.  New  York:  Springer. 

Conference  Presentations: 

Pruitt,  L.D.,  &  Luxton,  D.D.  (2015).  A  randomized  controlled  trial  of  home-based  tele-behavioral  health  care 
for  U.S.  military  personnel  and  veterans  with  depression.  Plenary  Presentation  at  the  2015  Military 
Health  System  Research  Symposium,  Fort  Lauderdale,  FL. 

Bounthavong,  M.,  Pruitt,  L.D.,  Smolenski,  D.J.,  Gahm,  G.,  Bansal,  A.,  Hansen,  R.N.  (2016).  Economic 

evaluation  of  in-home  telehealth  compared  to  in-person  treatment  delivery  for  managing  depression. 
International  Society  of  Pharmacoeconomics  and  Outcomes  Research,  21st  Annual  International 
Meeting. 

Conclusion 

This  study  is  the  first  randomized  controlled  trial  of  HBTBH  conducted  specifically  in  the  U.S.  military  setting. 
The  results  provide  important  information  about  the  feasibility,  safety,  and  clinical  efficacy  of  a  HBTBH 
treatment  that  can  inform  policy  decisions  about  the  expansion  of  behavioral  health  treatment  options  for  service 
members  and  Veterans.  The  overall  findings  of  this  study  support  the  feasibility  of  HBTMH  for  MHS  and  VHA 
beneficiaries.  Safety  procedures  were  successfully  implemented,  and  there  was  not  any  evidence  of  clinical 
worsening  in  the  in-home  condition  to  suggest  that  in-home  care  was  less  safe  than  traditional  in-office  care. 

The  results  of  the  present  trial  did  not,  however,  demonstrate  non-inferiority  of  HBTBH  compared  to  in-person 
treatment  based  on  BHS  and  BDI-II  scores.  The  post  hoc  analysis  that  used  a  95%  confidence  interval  showed 
the  in-home  treatment  modality  was  inferior  and  not  non-inferior  according  to  the  scores  from  the  BHS.  Also, 
because  the  95%  confidence  interval  for  the  comparison  of  the  BDI-II  included  zero,  we  cannot  make  a  strong 
conclusion  about  the  inferiority  of  the  treatment  on  this  measure.  We  therefore  cannot  conclude  from  the  non¬ 
inferiority  analyses  that  in-home  BATD  is  as  safe  as  in-office  care  based  solely  on  the  relative  differences  in 
BHS  and  BDI  scores  between  these  treatment  conditions.  It  is  important  to  emphasize  that  significant  reductions 
in  depression  symptoms  and  hopelessness  were  observed  across  both  groups.  Similar  improvement  was  also 
observed  on  measures  of  PTSD  symptoms  and  anxiety.  While  the  absence  of  a  non-treatment  control  (i.e.  a 
waitlist  control)  precludes  examination  of  whether  improvement  was  directly  attributable  to  treatment,  the 
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overall  improvement  in  clinical  outcomes  suggests  that  at  least  part  of  the  improvement  was  due  to  the  BATD 
treatment. 


The  results  also  provide  useful  infonnation  about  the  technical  and  logistical  feasibility  of  HBTBH  in  the 
military  setting.  The  legitimate  need  to  protect  government  infonnation  systems  while  connecting  to  external 
computer  systems  and  commercial  Internet  service  providers  can  create  technical  issues  that  need  to  be 
addressed  before  widespread  implementation.  This  issue  is  of  relevance  to  any  healthcare  system  that  utilizes  a 
secured  network.  While  the  present  evaluation  provided  a  level  of  control  over  the  technical  aspects  (supplied 
lap-top  and  camera),  real-world  applications  of  HBTBH  need  to  consider  use  of  personally-owned  equipment. 
Also,  clinical  appointments  were  conducted  during  standard  work  hours  (between  the  hours  of  8:00am  and 
3:00pm,  Monday  through  Friday).  While  these  hours  are  typical  for  standard  care  in  the  VHA  and  MHS  and 
were  appropriate  for  the  execution  of  the  study,  this  type  of  schedule  does  not  capture  the  added  flexibility  and 
conveniences  of  HBTBH.  One  of  the  many  benefits  of  in-home  care  is  the  ability  to  provide  clinical  services 
without  the  patient  having  to  depart  from  their  home  environment.  In  the  present  study,  the  military  participants 
were  very  often  at  work  before  their  appointment,  went  home  for  the  appointment,  and  sometimes  returned  to 
work  following  their  appointment.  Future  TBH  programs,  such  as  through  the  MHS’s  Telebehavioral  Health 
Clinics,  could  employ  providers  on  various  shifts  (after  hours  or  in  different  time  zones)  so  that  care  can  be 
provided  more  efficiently  to  patients  while  at  home. 

Telebehavioral  health  services  provided  to  the  home  or  other  locations  have  the  potential  to  address  current  and 
future  health  needs  of  military  service  members  and  Veterans,  especially  for  those  who  live  in  rural  or 
underserved  areas.  The  benefits  of  home-based  care  also  extend  to  family  members  or  other  caregivers  who  may 
otherwise  share  the  financial  burden  and  inconvenience  of  assisting  with  travel  to  health  care  services.  The 
value  of  home-based  treatment  lies  in  its  potential  to  increase  access  to  care  for  a  population  that  has  a  well- 
documented  history  of  low  treatment  seeking  behavior  and  other  barriers  to  overcome  (e.g.,  stigma,  frequent 
relocations,  deployment,  and  highly  demanding  work  duty).  Home-based  care  is  a  viable  option,  especially 
when  traditional  in-office  care  is  less  practical. 
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OBJECTIVES:  Despite  the  growing  evidence  supporting  Home-based  telebehavioral  health  (HBTBH) 
in  improving  mental  health,  there  has  not  been  an  economic  analysis  comparing  it  to  traditional  In- 
person  (IP)  setting.  The  National  Center  for  Telehealth  and  Technology  recently  completed  a 
randomized  controlled  trial  comparing  the  safety  and  efficacy  of  HBTBH  among  U.S.  military 
personnel  and  Veterans  with  depression.  We  performed  a  trial-based  economic  analysis  comparing 
HBTBH  and  IP  care  for  depression. 

METHODS:  We  performed  a  cost-minimization  analysis  on  121  patients  with  depression  to  assess 
the  economic  impact  of  HBTBH  versus  IP  behavioral  health  care  from  the  payer  perspective 
(Department  of  Defense  and  Veterans  Affairs)  at  3  months.  Sample  population  included  US  service 
members  and  veterans  with  a  diagnosis  of  depression  from  Joint  Base  Lewis-McChord  (Washington 
state)  and  veterans  were  enrolled  from  the  VA  Health  Care  System  (Portland,  Oregon).  We  also 
performed  a  scenario  analysis  assuming  that  all  patients  had  access  to  personal  video  conferencing 
technology.  One-way  and  probabilistic  sensitivity  analyses  were  performed  to  test  the  robustness  of 
the  model  assumptions. 

RESULTS:  In  the  base-case  analysis  the  total  direct  cost  of  HBTBH  care  was  higher  than  IP  care 
($68,483  versus  $21 ,561 ).  This  translates  to  an  $81 3  increase  per  patient.  Assuming  that  patients 
personally  owned  computers  could  be  used,  HBTBH  care  was  less  costly  compared  to  IP  care 
($18,242  versus  $21,561).  In  one-way  sensitivity  analyses,  the  proportion  of  patients  using  personal 
computers  was  a  major  driver  of  direct  costs.  Probabilistic  sensitivity  analysis  did  not  differ 
substantially  from  the  base-case  results. 

CONCLUSIONS:  Total  direct  cost  of  HBTBH  care  was  sensitive  to  whether  patients  were  able  to 
utilize  their  personally  owned  technology.  Using  patients  existing  in  home  video  conferencing 
technology  accrues  lower  costs  while  maintaining  healthcare  quality.  Health  care  policies  centered  on 
implementation  of  HBTBH  care  may  benefit  from  an  initial  assessment  of  technology  coverage 
among  beneficiaries. 
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Abstract 

Objective:  Evidence  of  feasibility,  safety,  and  effectiveness  of  home-based  telebehavioral  health 
(HBTBH)  needs  to  be  established  before  adoption  of  HBTBH  in  the  military  health  system 
(MHS)  can  occur.  The  purpose  of  this  randomized  controlled  non-inferiority  trial  was  to  compare 
the  safety,  feasibility,  and  effectiveness  of  HBTBH  to  care  provided  in  the  traditional  in-office 
setting  among  military  personnel  and  Veterans.  Method:  One  hundred  and  twenty  one  U.S. 
military  service  members  and  Veterans  were  recruited  at  a  military  treatment  facility  and  a 
Veterans  Health  Administration  (VHA)  hospital.  Participants  were  randomized  to  receive  eight 
sessions  of  behavioral  activation  treatment  for  depression  (BATD)  either  in  the  home  via 
videoconferencing  (VC)  or  in  a  traditional  in-office  (same-room)  setting.  Participants  were 
assessed  at  baseline,  mid-treatment  (4  weeks),  post-treatment  (8  weeks)  and  3  months  post 
treatment.  Results:  Mixed-effects  modeling  results  with  Beck  Hopelessness  Scale  (BHS)  and 
Beck  Depression  Inventory  II  (BDI-II)  scores  suggested  relatively  strong  and  similar  reductions 
in  hopelessness  and  depressive  symptoms  for  both  groups;  however,  non-inferiority  analyses 
failed  to  reject  the  null  hypothesis  that  in-home  care  was  no  worse  than  in-office  treatment  based 
on  these  measures.  There  were  not  any  differences  found  between  treatment  groups  in  regards  to 
treatment  satisfaction.  Safety  procedures  were  successfully  implemented,  supporting  the 
feasibility  of  home-based  care.  Conclusions:  BATD  can  be  feasibly  delivered  to  the  homes  of 
active  duty  service  members  and  Veterans  via  VC.  Small  group  differences  suggest  a  slight 
benefit  of  in-person  care  over  in-home  telehealth  on  some  clinical  outcomes.  Reasons  for  this 
are  discussed. 


Keywords:  Telemedicine;  Telemental  health;  Depression;  Military;  Veterans 
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Home-based  telebehavioral  health  (HBTBH)  is  the  provision  of  behavioral  health 
services  directly  to  a  patient’s  home  with  the  use  of  telecommunications  technologies.  This 
method  of  providing  care  has  the  potential  to  benefit  military  personnel,  veterans,  and  their 
families  by  improving  access  to  care  and  by  reducing  the  barriers  that  may  impede  treatment¬ 
seeking  and  engagement  in  care.  A  recent  report  from  the  RAND  Corporation  (Brown  et  al., 
2015)  notes  that  there  are  more  than  300,000  U.S.  military  service  members  and  1  million  family 
dependents  in  geographically  remote  locations  that  may  benefit  from  telehealth  services. 
Behavioral  health  care  services  provided  directly  in  the  home  can  reduce  or  eliminate  travel 
burden  to  treatment  facilities  and  may  be  especially  convenient  for  service  members  and  veterans 
who  have  limited  mobility  as  the  result  of  co-occurring  chronic  health  conditions  or  physical 
impairments  (Luxton,  Pruitt,  O’Brien,  &  Kramer,  2015).  The  option  to  receive  care  in  the 
privacy  of  one’s  own  home  may  also  ease  apprehension  among  military  personnel  who  are 
concerned  about  stigma  associated  with  seeking  mental  health  care  (Egede  et  al.,  2009;  Pruitt, 
Luxton,  &  Shore,  2014).  These  potential  benefits  of  HBTBH  have  been  recognized  for  some 
time  within  the  US  Department  of  Defense  (DoD)  the  military  health  system’s  (MHS)  model  of 
care  (Mullen,  2010),  yet  HBTBH  is  not  presently  recognized  as  a  standard  of  care  in  the  MHS. 

The  U.S.  Department  of  Veteran’s  Health  Affairs  (VHA)  has  been  a  leader  in 
establishing  and  expanding  telebehavioral  health  (TBH)  service  options  for  Veterans  (Darkins, 
Ryan,  Kobb,  Foster,  Edmonson,  et  al,  2009;  Strachan  et  al.,  2012).  The  feasibility  and 
effectiveness  of  TBH  has  been  supported  by  several  pilot  studies  and  randomized  controlled 
trials  (for  a  review  see  Ferrer,  Parish,  Johnston,  Callahan,  &  Yellowlees,  2012).  The  majority  of 
published  TBH  studies  involving  military  veterans  have  focused  on  the  treatment  of  post- 
traumatic  stress  disorder  (PTSD)  with  evidence-based  treatments  including  prolonged  exposure 
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therapy  (Tuerk,  Yoder,  Ruggiero,  Gros,  &  Acierno,  2010;  Gros,  Yoder,  Tuerk,  Lozano,  & 
Aciemo,  2011),  Cognitive  Processing  Therapy  (Morland,  Hynes,  Mackintosh,  Resick,  &  Chard, 
2011;  Morland,  Mackintosh,  Greene,  Rosen,  Chard,  Resick,  et  al,  2014),  or  group  anger 
management  therapy  (Morland,  Greene,  Rosen,  Foy,  Reilly,  Shore  et  al,  2010),  delivered  to 
small  outpatient  clinics  with  less  staffing  resources.  Other  studies  have  specifically  evaluated 
TBH  treatments  delivered  to  the  homes  of  veterans  (Egede  et  al.,  2009;  Yuen,  Gros,  Price, 
Zeigler,  Tuerk,  Foa,  &  Acierno,  2015).  For  example,  Egede  and  colleagues  (2009)  conducted  an 
RCT  that  evaluated  the  effectiveness  of  behavioral  activation  treatment  for  depression  (BATD; 
Lejuez,  Hopko,  &  Hopko,  2001)  to  treat  elderly  veterans  with  major  depressive  disorder  either 
in-home  with  videoconferencing  (VC)  technology  or  by  traditional  in-person  services.  The 
results  showed  that  the  treatment  delivered  by  in-home  video  teleconference  was  as  effective  as 
by  face-to-face  therapy  sessions.  The  VHA  has  also  expanded  its  HBTBH  program  through 
larger  pilot  programs  at  the  national  level  (Godleski,  Darkins,  &  Peters,  2012). 

While  the  expansion  of  HBTBH  options  in  the  VHA  is  promising,  questions  remain 
about  the  feasibility,  safety,  and  effectiveness  of  home-based  TBH  treatments  in  the  military 
setting.  Particular  questions  concern  the  feasibility  of  using  the  military’s  information 
technologies  (IT)  infrastructure  for  VC  and  the  applicability  of  general  HBTBH  procedures 
including  safety  and  emergency  management  protocols.  The  safety  of  HBTBH  is  a  primary  issue 
to  be  evaluated  given  that  care  provided  to  settings  without  clinical  staff  on-site  (e.g.,  the  home) 
require  additional  safety  planning  and  procedures  to  manage  safety  risks  (Luxton,  O’Brien, 
McCann,  &  Mishkind,  2012).  Principal  concerns  involve  how  to  appropriately  plan  for  and 
manage  risks  including  psychiatric  emergencies  (i.e.,  worsening  of  clinical  symptoms,  suicidal 
behavior)  and  medical  emergencies  that  could  occur  without  clinical  staff  on-site  (Luxton, 
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Sirotin,  &  Mishkind,  2010).  There  are  no  published  trials  with  active-duty  military  personnel  that 
have  evaluated  whether  treatments  provided  via  VC  in  the  home  are  comparable  in  clinical 
efficacy  to  traditional  in-office  care.  A  preliminary  pilot  study  evaluation  (Luxton,  Pruitt, 
O’Brien,  &  Kramer,  in  press)  tested  the  basic  procedures  and  supported  the  technical  feasibility 
of  home-based  care  in  the  military  setting.  However,  a  direct  comparison  of  HBTBH  to 
traditional  in-office  care  is  thus  needed  to  inform  policy  decisions  regarding  the  adoption  and 
expansion  of  HBTBH  within  the  MHS  and  to  add  to  the  growing  body  of  research  on  the 
effectiveness  of  TBH  across  diverse  populations. 

The  primary  aim  of  this  trial  was  to  evaluate  the  safety  and  feasibility  of  providing 
military  service  members  and  veterans  with  HBTBH  care  by  comparing  it  to  conventional  in- 
office  care.  We  selected  BATD  (Lejuez,  Hopko,  Acierno,  Daughters,  &  Pagoto,  201 1)  as  the 
target  treatment  for  the  present  trial  given  that  depression  is  one  of  the  most  prevalent  psychiatric 
conditions  among  Veterans  and  military  personnel  and  it  is  the  most  frequent  diagnosis 
associated  with  psychiatric  hospitalization  in  both  the  active  and  reserve  components  of  the  U.S. 
Anned  Forces  (Armed  Forces  Health  Surveillance  Center,  2013;  Bagalman,  2013).  Prevalence 
rates  for  depression  based  on  screening  data  have  been  estimated  to  be  12%  among  deployed 
service  members  (Gadermann  et  ah,  2012)  and  20%  among  Operation  Iraqi  Freedom/Operation 
Enduring  Freedom  veterans  (Seal  et  ah,  2009;  Corson  et  ah,  2013).  Depression  also  frequently 
co-occurs  with  other  conditions,  such  as  PTSD  and  physical  injuries  among  military  personnel 
and  Veterans  and  can  slow  recovery  and  return  to  duty  among  those  with  comorbid  conditions 
(Hoge,  Auchterlonie,  &  Milliken,  2006;  Hoge  et  ah,  2004;  Milliken,  Auchterlonie,  &  Hoge, 
2007).  BATD  aims  to  reengage  depressed  individuals  in  their  lives  through  focused,  values- 
based  activation  strategies.  These  strategies  are  intended  to  counter  patterns  of  negative  affect, 
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inactivity,  and  withdrawal  by  reestablishing  naturalistically  reinforcing  behaviors  which  in  turn 
alleviate  depressed  mood  and  create  stable  patterns  of  activity  and  engagement.  BATD  is  also 
presumed  to  be  a  compatible  treatment  for  military  personnel  given  that  it  is  a  problem-focused, 
direct  intervention  that  may  be  perceived  as  less  stigmatizing  than  other  “emotion-focused” 
treatments  (Egede,  Frueh,  Richardson,  et  al,  2009). 

A  non-inferiority  design  was  used  for  this  trial  because  it  is  well-suited  for  comparing 
novel  adaptations  of  established  treatments  that  have  demonstrated  efficacy  (Greene,  Morland, 
Durkalski,  &  Frueh,  2008).  BATD  has  a  strong  empirical  evidence-base  (Cuijpers,  van  Straten, 
Andersson,  &  van  Oppen,  2008)  and  it  has  been  evaluated  in  prior  TBH  studies  with  veteran 
populations  (Egede,  et  al,  2009).  Both  active  duty  service  members  and  veterans  were  included 
to  provide  preliminary  data  regarding  any  unique  differences  between  these  populations  and 
treatment  settings.  We  hypothesized  that  in-home  care  would  be  no-worse  than  in-office  care  on 
clinical  measures  that  reflect  safety  and  clinical  outcomes  (measures  of  hopelessness  and 
depression  scores)  and  that  participants  in  both  groups  would  show  improved  hopelessness  and 
depression  scores  at  treatment  end.  We  also  predicted  that  BATD  delivered  in  home  would  be  as 
safe  as  in-office  treatment.  Our  assessment  of  safety  additionally  consisted  of  both  qualitative 
analysis  of  any  safety  events  requiring  activation  of  our  safety  protocol  as  well  as  the  evaluation 
of  the  relative  clinical  efficacy  of  HBTBH. 

The  study  was  approved  by  the  Institutional  Review  Boards  at  Madigan  Army  Medical 
Center  and  the  VA  Portland  HCS  and  reviewed  by  the  Army  Human  Research  Protection  Office. 
The  trial  was  partially  funded  by  the  Military  Operational  Medicine  and  Research  Program  and  is 
registered  on  the  United  States  National  Institutes  of  Health  Clinical  Trials  Registry, 
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(ClinicalTrials.gov  Identifier  #NCT01599585)  available  online  at: 
http  ://clinicaltrials  .gov/ show/NCTO  1599585. 

Methods 

Participants 

A  complete  description  of  the  trial  methodology  is  available  in  a  previous  publication 
(Luxton  et  al,  2014).  Participants  were  recruited  at  a  large  regional  military  treatment  facility 
located  at  Joint  Base  Lewis-McChord  (JBLM)  in  Washington  State  and  at  the  VA  Health  Care 
System  in  Portland,  Oregon  (VAPHCS).  Study  patients  at  JBLM  comprised  active-duty,  reserve, 
and  National  Guard  service  members.  The  study  patients  at  the  VAPHCS  site  were  U.S.  military 
Veterans  receiving  health  care  services  through  the  VA  hospital. 

Participants  were  included  in  the  study  if  they  met  Diagnostic  and  Statistical  Manual  of 
Mental  Disorders  (4th  ed.,  text  rev.;  DSM-IV-TR;  American  Psychiatric  Association,  2000) 
diagnostic  criteria  for  major  depressive  disorder  or  minor  depressive  disorder  based  on  the 
Structured  Clinical  Interview  for  the  DSM-IV  Axis  I  Disorders,  Research  Version,  Patient 
Edition  (SCID-I/P;  First,  Spitzer,  Gibbon,  &  Williams,  2002).  The  inclusion  of  both  major  and 
minor  depressive  disorder  in  the  trial  provided  for  a  more  generalizable  representation  of  patients 
typically  seen  at  the  sites.  Clinical  assessors  were  unaware  of  treatment  condition  throughout  the 
trial.  All  participants  had  to  have  access  to  high  speed  Internet  available  in  their  homes.  The  full 
study  inclusion  and  exclusion  criteria  are  shown  in  Table  2. 

Procedures 

Participant  recruitment  began  in  August  2012  and  concluded  in  July  2014.  Participants  at 
both  study  sites  were  referred  to  the  study  by  mental  health  care  providers  from  outpatient 
mental  health  clinics  as  well  as  by  self-referral  (research  flyers  were  placed  in  common  areas 
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around  the  hospital  facilities).  A  study  research  coordinator  conducted  initial  eligibility  criteria 
screening  via  telephone.  Eligible  participants  were  then  scheduled  to  complete  informed  consent 
procedures  and  an  in-person  interview  (i.e.  the  SCID-I/P)  with  the  study  team’s  independent 
clinical  assessor.  Patients  at  the  PVA  site  were  given  $20  for  each  of  the  first  three  assessment 
visits  and  $40  for  completion  of  3  month  follow-up  assessment.  Patients  at  the  JBLM  site  were 
not  compensated  for  their  time  per  US  Army  regulations.  Active  Duty  participants  were  provided 
with  medical  appointment  permission  slips  to  allow  them  time  to  attend  treatment  sessions 
outside  of  normal  duties,  including  time  to  return  home  if  randomized  to  the  in-home  condition. 

The  CONSORT  chart  in  Figure  1  shows  the  progression  of  participants  through  the  study 
phases.  A  total  of  92  participants  met  all  inclusion  criteria  and  were  randomized  at  the  military 
site  and  29  were  randomized  at  the  VA  site.  Participants  were  randomized  to  treatment  condition 
in  a  1:1  ratio  using  block  randomization  of  size  ten,  stratified  by  baseline  major  or  minor 
depressive  disorder  diagnosis  using  Stata  12.1  (StataCorps,  2013).  A  suicide  assessment  and  risk 
management  standard  operating  procedure  (SOP)  used  at  Madigan  Army  Medical  Center  was 
used  to  assess  and  document  current  ideation,  presence  of  a  plan,  suicidal  intent,  history  of 
previous  attempts  and  degree  of  impulsivity.  Risk  correlates  (e.g.,  recent  loss,  financial 
problems),  preparatory  behavior  (e.g.,  available  means),  and  other  risk  factors  (e.g.,  substance 
dependence)  were  also  assessed  and  documented.  The  SOP  was  administered  at  the  baseline 
assessment,  the  first  treatment  session,  and  re-administered  at  each  subsequent  session  if  a 
patient  endorsed  current  elevated  risk  per  the  SOP. 

Participants  randomized  to  the  in-home  condition  were  provided  with  a  Dell  M6500 
laptop  computer  and  Tandberg  Precision  High  Definition  webcam.  The  lap-tops  were  pre-loaded 
with  Jabber  video  software  (Cisco  Systems).  This  software  was  selected  because  its  level  of 
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security  and  encryption  is  approved  for  use  by  the  U.S.  Army  and  the  VAPHCS.  Participants 
randomized  to  the  in-home  condition  were  scheduled  for  a  brief  test  of  the  VC  software  and 
equipment  prior  to  initiation  of  the  first  treatment  session. 

Treatment  Protocol 

An  eight-session  BATD  protocol  that  has  been  successfully  delivered  both  in-person  and 
VC  in  the  past  was  used  as  the  study’s  intervention  (Cuijpers  et  al,  2008).  Session  one  was 
devoted  to  psychoeducation,  the  treatment  rationale,  and  introduction  of  the  concept  of  daily 
activity  monitoring.  Session  two  focused  on  identifying  goals/values  across  five  different  major 
life  domains  (i.e.,  relationships,  education/career,  recreation/interests,  mind/body/spirituality,  and 
daily  responsibilities).  Sessions  3  through  8  were  focused  on  activity  planning  consistent  with 
these  goals/values.  Activity  planning  is  the  process  of  collaboratively  selecting  and  scheduling 
activities  consistent  with  one’s  values  that  have  a  high  likelihood  of  invoking  positive  mood  or 
functional  outcomes  (i.e.,  are  reinforcing  to  the  individual).  For  example,  if  a  patient  identifies 
commitment  to  his/her  marriage  as  a  value,  specific  activities  may  involve  scheduling  a  ‘date 
night,’  stopping  to  buy  flowers  on  the  way  home  from  work,  and  helping  with  the  dishes  after 
dinner. 

Both  groups  received  the  same  BATD  treatment  for  50-60  minute  sessions  every  week 
for  8  weeks  with  clinical  assessments  conducted  at  baseline,  4-week  midpoint,  8-week  treatment 
completion,  and  3 -month  follow-up.  There  were  five  treatment  providers  (4  at  JBLM  and  1  at 
PVA),  all  were  doctoral  level  mental  health  providers  who  received  training  from  a  BATD 
expert  and  consultant..  Fidelity  review  was  conducted  by  the  BATD  expert  who  reviewed  video¬ 
recordings  of  a  randomly  selected  10%  of  all  treatment  sessions.  The  fidelity  reviews  confirmed 
an  overall  adherence  of  98.19%  for  all  clinicians  providing  treatment. 
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Measures 

The  Beck  Hopelessness  Scale  (BHS;  Beck,  Weissman,  Lester,  &  Trexler,  1974)  and  the 
Beck  Depression  Inventory  II  (BDI-II;  Beck,  Streer,  &  Brown,  1996)  served  as  the  primary 
outcomes  for  the  non-inferiority  objectives  of  the  trial.  The  strong  association  between 
hopelessness  and  suicide  risk  (Beck,  Brown,  Berchick,  Stewart,  &  Steer,  1990)  made 
measurement  of  hopelessness  a  viable  indicator  of  safety.  The  BDI-II  also  provides  for  an 
assessment  of  safety  as  significant  symptom  worsening  could  reflect  adverse  effects  of  the 
treatment  modality.  Other  clinical  outcomes  including  depression  diagnosis,  PTSD  severity,  and 
anxiety  severity  were  analyzed  as  secondary  outcomes  along  with  assessment  of  technical  and 
safety  events  during  the  trial.  Safety  events  were  not  anticipated  to  occur  with  sufficient 
regularity  to  allow  statistical  hypothesis  testing.  All  of  the  outcome  measures  and  checklists 
administered  in  the  trial  are  described  below.  Estimates  of  internal  consistency  reliability  for 
scale  measures,  by  assessment  time  and  treatment  group,  are  provided  in  the  tables. 

BIIS  (Beck,  Weissman,  Lester,  &  Trexler,  1974) 

The  BHS  comprises  20  true-false  statements  relating  to  feelings  of  hopelessness  about  the 
future.  After  reverse  scoring  several  items,  sum  scores  are  calculated  with  a  possible  range  of  0 
to  20,  with  higher  scores  indicating  higher  levels  of  hopelessness. 

BDI-II  (Beck,  Streer,  &  Brown,  1996) 

The  BDI-II  comprises  2 1  items  with  four  response  categories  for  each  item.  Sum  scores 
are  calculated  with  a  possible  range  of  0  to  63,  with  higher  scores  indicating  higher  levels  of 
depression  symptom  severity. 

SCID-I/P  (First  et  al,  2002) 

The  SCID-I/P  was  used  for  initial  diagnostic/screening  purposes  and  at  follow-up 
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assessments  for  the  presence  of  MDD.  The  SCID-I/P  has  excellent  established  inter-rater 
reliability  (overall  kappa  =  0.85). 

Beck  Anxiety  Inventory  (BAI;  Beck  &  Streer,  1990) 

The  BAI  is  a  self-report  measure  consisting  of  2 1  items  designed  to  discriminate  anxiety 
from  depression.  Similar  to  the  BDI-II,  the  sum  scores  range  from  0  to  63. 

PTSD  Checklist  -  Military  Version  (PCL-M;  Weathers,  Huska,  &  Keane,  1991) 

The  PCL-M  is  a  self-report  measure  that  evaluates  all  17  DSM-IV-TR  PTSD  symptoms 
across  the  three  primary  symptom  clusters  using  a  5-point  Likert-type  scale.  A  total  score  of  50 
typically  serves  as  the  threshold  for  identifying  probable  PTSD  among  those  reporting  military 
related  trauma(s). 

Inventory  of  Attitudes  Toward  Seeking  Mental  Health  Services  (IASMHS;  Mackenzie,  Knox, 
Gekoski,  &  Macaulay,  2004) 

The  IASMHS  is  a  24  item  assessment  of  help-seeking  attitudes.  It  includes  the  following 
three  factors  based  on  components  of  Ajzen’s  Theory  of  Planned  Behavior  (Ajzen,  1985): 
psychological  openness,  help-seeking  propensity,  and  indifference  to  stigma.  Test-retest 
reliability  for  the  factors  ranges  from  moderate  to  high.  Convergent  validity  has  been 
demonstrated  by  effectively  differentiating  those  who  would  and  would  not  use  services.  The 
IASMHS  was  completed  at  baseline,  post  treatment,  and  follow-up. 

Client  Satisfaction  Questionnaire  (CSQ;  Nguyen,  Attkisson,  &  Stegner,  1983) 

The  CSQ-8  is  an  8-item  self-report  measure  of  general  satisfaction  with 
psychotherapeutic  treatment.  Participants  are  asked  to  rate  satisfaction  on  a  4-point  scale,  with  a 
possible  range  of  8  to  32,  with  higher  scores  indicating  greater  satisfaction.  Internal  consistency 
and  construct  validity  have  been  established  and  the  measure  is  widely  used  in  research.  The 
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CSQ  was  administered  at  post  treatment. 

Treatment  Session  Checklist  (Luxton  etal.  2014) 

Safety  related  data  were  recorded  after  each  treatment  session  on  the  treatment  session 
checklist  (see  Luxton  et  al.  2014).  This  checklist  is  designed  to  collect  information  for  the 
evaluation  of  clinical  telehealth  sessions.  It  is  used  to  document  pertinent  safety  infonnation 
including  current  suicidal  ideation,  homicidal  ideation,  the  presence  of  a  firearm  at  the  patient’s 
location,  and  signs  of  intoxication,  disorientation,  and  severe  emotion  dysregulation.  There  are 
also  questions  related  to  the  in-home  environment,  such  as,  “Is  anyone  else  at  home  today?”  and 
“Do  you  feel  that  your  environment  is  safe  and  private?”  This  checklist  is  also  used  to  document 
telehealth  equipment  and  network  connectivity  status,  lighting,  and  any  disruptions  to  assess 
technical  feasibility. 

Demographic  questionnaire 

Participants  provided  demographic  infonnation  including  occupation/work 
status/income/living  situation,  branch  of  service/highest  rank,  pain  rating  (0-10),  and 
medications. 

Sample  size 

Sample  size  was  estimated  using  a  medium  effect  size  value  of  Cohen’s/2  =  0.15,  a  one- 
tailed  a  =  0.05,  and  (3  =  0.80  based  on  the  BHS.  This  resulted  in  a  requirement  of  54  subjects  per 
study  group.  Assuming  a  10%  attrition  rate,  this  number  was  adjusted  to  60  subjects  per  group 
with  up  to  150  participants  approved  by  the  IRB.  Prior  to  start  of  the  trial,  only  the  BHS  was 
specified  as  the  primary  outcome  for  the  non-inferiority  analyses.  Given  that  the  treatment  is  for 
depression,  the  BDI  was  respecified  from  a  secondary  outcome  to  a  primary  outcome  for 
additional  non-inferiority  analyses  prior  to  treatment  condition  unblinding  and  data  analysis.  The 
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non-inferiority  margin  (8)  was  identified  as  a  standardized  difference  of  0.50  (see  Luxton,  Pruitt, 
et  al,  2014).  A  recalculation  of  sample  size  for  this  outcome  measure  identified  that  with  8  = 

0.50,  an  expected  difference  between  groups  (0)  =  0,  a  one-sided  a  =  0.05,  and  (3  =  0.80,  49 
subjects  would  be  required  per  study  group  to  reject  the  null  hypothesis  that  any  differences  were 
greater  than  or  equal  to  8.  These  parameters  are  consistent  with  recommendations  for  non¬ 
inferiority  trials  (e.g.,  Mohr,  Ho,  Duffecy,  Reifler,  Sokol,  Burns,  et  al,  2012;  Nutt,  Allgulander, 
Lecrubier,  et  al  2008).  However,  Green  and  colleagues  (2008)  have  recommended  (3  =  0.90  and  a 
two-tailed  alpha  of  0.05,  which  we  considered  for  post-hoc  analysis.  The  original  target  sample 
size  was  not  changed. 

Statistical  Methods 

We  used  linear  mixed  effect  regression  models  to  estimate  differences  in  the  means  of  the 
primary  outcome  measures  over  time  (Singer  &  Willett,  2003).  We  included  treatment 
assignment  in  the  model  as  a  binary  indicator  and  measurement  time  as  a  4-level  nominal 
variable.  All  analyses  used  the  baseline  assessment  as  the  referent  time  point.  Statistical 
inference  on  the  difference  between  treatment  groups  at  all  assessment  times  post  baseline  was 
based  on  the  treatment  by  time  interaction  terms  included  in  the  models.  Primary  inference  was 
focused  on  the  post  assessment  measurement  time.  We  used  the  square  root  of  the  sum  of  the 
random  intercept  variance  and  the  residual  variance  as  the  estimate  of  baseline  variation  for  the 
purposes  of  standardization.  Rejection  of  the  null  hypothesis  was  indicated  by  the  upper  bound 
of  the  standardized  90%  confidence  interval  falling  below  0.50.  For  both  primary  outcomes,  we 
examined  an  intent-to-treat  and  a  per  protocol  model.  All  models  were  estimated  using  restricted 
maximum  likelihood  in  Stata  12.1  (StataCorps,  2013). 
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We  also  used  the  linear  mixed  effects  regression  model  to  analyze  the  secondary 
outcomes  of  the  BAI,  the  PCL-M,  and  the  IASMHS.  Because  these  measures  were  not 
hypothesized  as  measures  for  non-inferiority,  we  used  95%  percent  confidence  intervals  to 
evaluate  differences  between  the  treatment  groups.  For  the  SCID-I/P  assessment  of  major 
depressive  disorder,  the  outcome  used  in  analysis  was  a  binary  indicator  for  meeting  or  not 
meeting  the  criteria.  As  such,  we  used  a  population-averaged  logistic  model  to  compare  the 
treatment  groups  in  tenns  of  change  in  the  prevalence  of  the  diagnosis  over  time.  For  the  CSQ, 
we  used  a  Student’s  t-test  to  compare  the  means  at  post  treatment.  There  were  no  planned 
statistical  analyses  for  descriptive  data  on  the  occurrence  of  adverse  events  and  the  initiation  of 
the  safety  protocol  data  given  the  low  expected  rate  of  occurrence. 

Post  hoc  analyses.  Post  hoc  analyses  included  the  estimation  of  the  primary  outcome 
regression  models  with  demographic  covariates  and  with  restrictions  to  just  the  active  military 
sample  or  to  participants  with  an  initial  diagnosis  of  major  depressive  disorder  to  examine  the 
influence  of  sample  heterogeneity  on  inference.  We  also  examined  the  results  of  the  primary 
outcome  models  using  95%  confidence  intervals  to  determine  if  conclusions  of  treatment 
inferiority  were  warranted. 

Missing  data.  Missing  data  occurred  predominantly  in  a  mono  tonic  fashion  as  a  function 
of  treatment  withdrawal  or  loss  to  follow-up.  The  regression  models  described  above  allowed  us 
to  retain  all  participants  who  provided  data  at  baseline  irrespective  of  data  availability  at 
subsequent  assessment  times.  The  assumption  of  this  modeling  strategy  is  that  the  data  are  at 
least  missing  at  random.  We  used  a  logistic  regression  model  to  estimate  the  association  between 
dropout  and  a  vector  of  baseline  outcome  scores  and  demographic  factors  to  evaluate  this 
assumption.  As  a  sensitivity  analysis,  we  used  latent  curve  models  to  estimate  a  selection  model 
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of  dropout  that  assumes  data  missing  not  at  random  (Enders,  2010).  In  this  model,  we 
simultaneously  estimated  the  growth  curve  parameters  for  the  outcome  from  baseline  to  post-test 
using  a  linear  specification  of  change  over  time  and  logistic  models  of  dropout  at  the  mid  and 
post  assessment  times.  Antecedents  of  the  dropout  indicators  were  treatment  assignment,  the 
outcome  score  at  that  measurement  occasion,  and  the  outcome  scores  of  the  preceding 
measurement  occasion.  We  used  M plus  7.1  (Muthen  &  Muthen,  2012)  to  estimate  the  selection 
models. 

Results 

Study  completion 

A  total  of  40  participants  completed  all  eight  sessions  in  the  in-home  condition  and  42 
completed  all  eight  sessions  in  the  in-office  condition  for  a  total  attrition  rate  of  32.23%.  In  the 
in-home  condition,  there  were  16  withdrawals,  three  losses  to  follow-up,  and  three  who  did  not 
begin  treatment.  In  the  in-person  condition,  there  were  nine  withdrawals,  five  losses  to  follow¬ 
up,  and  three  who  did  not  begin  treatment.  The  difference  in  proportions  of  subjects  that  did  not 
complete  treatment  between  the  groups  was  not  statistically  significant  (in-home  =  35.48%,  in- 
person  =  28.81%,  x2  =  0.62,  df  =  1  ,p=  .433).  Baseline  scores  on  the  BHS  and  the  BDI-II  were 
not  associated  with  dropout.  None  of  the  demographic  variables  were  associated  with  dropout 
except  race;  however,  the  category  that  showed  the  strongest  association  with  dropout  was  the 
“other”  category  which  included  only  ten  participants.  This  evidence  suggested  that  missing  at 
random  was  a  reasonable  assumption  for  analysis. 

Primary  outcomes 

Table  3  displays  the  results  for  the  intent-to-treat  and  per  protocol  analyses  for  both  the 
BHS  and  the  BDI-II.  At  post  treatment,  participants  in  the  in-person  group  had  an  average 
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reduction  of  6.21  points  on  the  BHS  (95%  Cl  =  -7.38,  -5.05)  and  17.63  points  on  the  BDI-II 
(95%  Cl  =  -20.21,  -15.06).  Participants  in  the  in-home  group  had  an  average  reduction  of  3.91 
points  on  the  BHS  (95%  Cl  =  -5.25,  -2.57)  and  13.40  points  on  the  BDI-II  (95%  Cl  =  -16.36,  - 
10.44).  For  both  outcomes,  the  magnitude  of  decrease  over  time  was  less  pronounced  for  the  in- 
home  group  compared  to  the  in-person  group.  After  standardization  of  the  difference  between  the 
groups  at  post  assessment,  we  found  that  the  upper  bound  of  the  90%  confidence  interval 
included  0.50.  The  per  protocol  analysis  provided  similar  results;  however,  the  point  estimates 
were  somewhat  greater  in  magnitude  than  those  from  the  intent-to-treat  analysis.  See  Figures  3 
and  4  for  a  graphic  display  of  the  point  estimates  and  confidence  intervals  against  the  non¬ 
inferiority  margin. 

Secondary  outcomes 

Table  4  presents  the  results  of  the  analyses  of  the  additional  outcome  measures.  At 
baseline,  56  participants  (90.32%)  in  the  in-home  group  and  54  in  the  in-person  group  (91.53%) 
met  SCID-I/P  criteria  for  major  depressive  disorder.  At  post  treatment,  there  were  8  participants 
in  the  in-home  group  (17.78%)  and  6  in  the  in-person  group  (14.29%)  who  met  criteria  for  major 
depressive  disorder.  The  difference  in  reduction  of  the  number  of  participants  meeting  criteria  for 
major  depressive  disorder  was  not  statistically  significant  (b  =  0.33,  95%  Cl  =  -1.22,  1.88).  By 
the  twelve-week  follow-up,  4  participants  in  the  in-home  group  (9.52%)  and  8  in  the  in-person 
group  (22.22%)  met  criteria  for  major  Depressive  Disorder.  This  difference  was  also  not 
statistically  significant  (b  =  -0.89,  95%  Cl  =  -2.54,  0.77).  Participants  in  both  treatment  groups 
reported  reductions  in  anxiety  and  posttraumatic  stress  symptoms  and  in  mental  health  treatment 
stigma  as  measured  by  the  IASMHS.  There  were  no  statistically  significant  differences  between 
the  treatment  groups  on  these  outcomes.  Average  scores  on  the  CSQ  suggested  a  high  level  of 
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treatment  satisfaction  for  both  treatment  groups.  There  was  no  statistically  significant  difference 
between  the  treatment  groups  on  the  CSQ. 

Post  hoc  analysis 

We  did  not  identify  a  large  change  in  the  magnitude  of  the  differences  between  the 
treatment  groups  when  including  demographic  covariates  in  the  model  or  when  restricting  the 
analytic  sample  to  those  at  the  MTF  site  and  those  with  an  initial  major  depressive  disorder 
diagnosis.  This  suggested  that  the  conclusions  from  the  primary  models  were  robust  to  the 
heterogeneity  introduced  by  including  veterans  and  participants  with  minor  depression  into  the 
study  sample.  However,  given  the  small  number  of  veterans  and  participants  in  the  sample,  we 
were  not  able  to  formally  test  for  effect  measure  modification  in  the  association  between 
treatment  and  the  primary  and  secondary  outcomes. 

Using  the  95%  confidence  interval  (Cl)  as  opposed  to  the  90%  confidence  interval 
allowed  for  a  two-tailed  test  of  non-inferiority  and  inferiority.  The  95%  Cl  for  the 
unstandardized  difference  on  the  BHS  ranged  from  0.33  to  4.20  for  the  intent-to-treat  analysis 
and  from  0.57  to  4.67  for  the  per  protocol  analysis,  both  of  which  suggested  that  the  in-home 
method  of  delivery  was  inferior  to  the  in-person  method  of  delivery.  The  post  hoc  power  for  the 
intent-to-treat  analysis  was  0.81.  The  95%  confidence  intervals  for  the  unstandardized 
differences  on  the  BDI  ranged  from  -0.03  to  8.50  and  from  0.40  to  9.46  for  the  intent-to-treat  and 
per  protocol  analyses,  respectively.  Since  the  confidence  interval  for  the  intent-to-treat  analysis 
covered  0,  the  results  were  inconclusive  as  to  inferiority  of  the  in-home  method  of  delivery  on 
treatment  efficacy.  The  post  hoc  power  for  the  intent-to-treat  analysis  of  the  BDI-II  was  0.69. 

The  results  of  the  selection  model  are  presented  in  Table  5.  Dropout  by  the  mid  and  post 
treatment  assessments  was  not  associated  with  treatment  assignment,  prior  outcome  scores,  or 
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the  current  outcome  scores.  Estimates  of  the  treatment  differences  at  mid  and  post  treatment  for 
the  BHS  and  the  BDI-II  were  similar  to  those  presented  in  Table  3  for  the  intent- to-treat  models. 
These  findings  further  supported  an  assumption  of  data  at  least  missing  at  random. 

There  were  seven  participants  who  had  adverse  events  that  required  reporting  in  the  in- 
home  group  and  four  in  the  in-person  group.  None  of  these  adverse  events  (e.g.  a  severe 
exacerbation  of  asthma  symptoms)  were  detennined  to  be  related  to  study  procedures.  The  safety 
protocol  was  initiated  one  time;  this  occurred  for  a  military  service  member  in  the  in-home 
condition  who  contacted  his  study  provider  and  presented  in-person  to  the  research  staff.  The 
participant  reported  that  he  was  experiencing  distress  and  had  underreported  baseline  level  of 
suicidal  ideation  during  the  intake  assessment.  The  patient  was  assessed  by  a  supervisory 
psychologist  and  escorted  to  the  emergency  department  for  further  evaluation  as  per  the 
established  safety  protocol.  Additional  description  of  suicide  risk  management  used  in  the  trial  is 
provided  by  Luxton  et  al.  (2014). 

Of  the  378  treatment  sessions  completed  in  the  in-home  group,  there  were  190  (50.26%) 
VC  connectivity  issues  including  the  inability  to  initiate  a  webcam  connection  (n  =  137,  36.34%) 
and  the  inability  to  maintain  a  webcam  connection  once  initiated  (n  =  66,  20.3 1%).  Overall,  135 
(35.71%)  treatment  sessions  required  a  phone  call  to  resolve  a  technical  issue  or  to  complete  all 
or  a  portion  of  the  session’s  activities  because  of  technical  problems. 

Discussion 

This  study  is  the  first  randomized  controlled  trial  of  HBTBH  conducted  specifically  in 
the  US  military  setting.  The  results  provide  important  infonnation  about  the  feasibility,  safety, 
and  clinical  efficacy  of  a  HBTBH  treatment  that  can  inform  policy  decisions  about  the  expansion 
of  behavioral  health  treatment  options  for  service  members  and  for  Veterans.  The  overall 
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findings  of  this  study  support  the  feasibility  of  HBTMH  for  MHS  and  VHA  beneficiaries.  Safety 
procedures  were  successfully  implemented  and  there  was  not  any  evidence  of  clinical  worsening 
in  the  in-home  condition  to  suggest  that  in-home  care  was  less  safe  than  traditional  in-office  care. 

The  results  of  the  present  trial  did  not,  however,  demonstrate  non-inferiority  of  HBTBH 
compared  to  in-person  treatment  based  on  BHS  and  BDI-II  scores.  The  post  hoc  analysis  that 
used  a  95%  confidence  interval  showed  the  in-home  treatment  modality  was  inferior  and  not 
non -inferior  according  to  the  scores  from  the  BHS.  Also,  because  the  95%  confidence  interval 
for  the  comparison  of  the  BDI-II  included  zero,  we  cannot  make  a  strong  conclusion  about  the 
inferiority  of  the  treatment  on  this  measure.  We  therefore  cannot  conclude  from  the  non¬ 
inferiority  analyses  that  in-home  BATD  is  as  safe  as  in-office  care  based  solely  on  the  relative 
differences  in  BHS  and  BDI  scores  between  these  treatment  conditions.  It  is  important  to 
emphasize  that  significant  reductions  in  depression  symptoms  and  hopelessness  were  observed 
across  both  groups.  Similar  improvement  was  also  observed  on  measures  of  PTSD  symptoms, 
and  anxiety.  While  the  absence  of  a  non-treatment  control  (i.e.,  a  waitlist  control)  precludes 
examination  of  whether  improvement  was  directly  attributable  to  treatment,  the  overall 
improvement  in  clinical  outcomes  suggests  that  at  least  part  of  the  improvement  was  due  to  the 
BATD  treatment. 

The  present  study  has  several  limitations  that  could  be  addressed  in  future  evaluations  of 
HBTBH.  The  sample  size  was  insufficient  to  fully  address  the  questions  of  non-inferiority  and 
inferiority/superiority  of  HBTMH.1  Future  evaluations  would  benefit  from  larger  samples  based 
on  an  established  non-inferiority  margin  and  consideration  of  trivial  differences  in  point 
estimates.  The  relatively  low  depression  scores  of  both  groups  at  baseline,  combined  with  high 
variability  among  participants  may  have  also  influenced  the  findings.  Also,  the 
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inclusion/exclusion  criteria  limited  the  level  of  risk  of  the  enrolled  participants  and  the  study  was 
not  powered  to  detect  a  sufficient  number  of  safety  events  to  make  a  statistical  inference  about 
safety  solely  based  on  the  number  of  safety  events.  Much  larger  program  evaluations  of  HBTBH 
provide  an  opportunity  to  further  examine  factors  associated  with  patient  safety  and  clinical 
outcomes  with  more  diverse  clinical  profiles.  The  use  of  standardized  metrics,  such  as  those 
proposed  by  the  American  Telemedicine  Association  (Shore  et  al,  2014),  are  recommended  to 
standardize  data  collected  by  clinics  across  the  MHS  including  clinical  outcomes,  patient 
satisfaction  and  preferences  for  TBH  care. 

The  results  also  provide  useful  information  about  the  technical  and  logistical  feasibility  of 
HBTBH  in  the  military  setting.  The  legitimate  need  to  protect  government  information  systems 
while  connecting  to  external  computer  systems  and  commercial  Internet  service  providers  can 
create  technical  issues  that  need  to  be  addressed  before  widespread  implementation.  This  issue  is 
of  relevance  to  any  healthcare  system  that  utilizes  a  secured  network.  While  the  present 
evaluation  provided  a  level  of  control  over  the  technical  aspects  (supplied  lap-top  and  camera), 
real-world  applications  of  HBTBH  need  to  consider  use  of  personally  owned  equipment  (Luxton, 
et  al.  in  press).  Also,  clinical  appointments  were  conducted  during  standard  work  hours 
(between  the  hours  of  8:00am  and  3:00pm,  Monday  through  Friday).  While  these  hours  are 
typical  for  standard  care  in  the  VHA  and  MHS,  and  were  appropriate  for  the  execution  of  the 
study,  this  type  of  schedule  does  not  capture  the  added  flexibility  and  conveniences  of  HBTBH. 
One  of  the  many  benefits  of  in-home  care  is  the  ability  to  provide  clinical  services  without  the 
patient  having  to  depart  from  their  home  environment.  In  the  present  study,  the  military 
participants  were  very  often  at  work  before  their  appointment,  went  home  for  the  appointment, 
and  sometimes  returned  to  work  following  their  appointment.  Future  TBH  programs,  such  as 
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through  the  MHS’s  Telebehavioral  Health  Clinics,  could  employ  providers  on  various  shifts 
(after  hours  or  in  different  time  zones)  so  that  care  can  be  provided  more  efficiently  to  patients 
while  at  home. 

The  present  study  adds  to  other  research  regarding  the  applicability  of  evidence-based 
treatments  delivered  via  telehealth  technologies.  A  growing  literature  base  of  studies  have 
generally  supported  the  feasibility  and  effectiveness  of  providing  behavioral  health  treatments  to 
the  home  or  other  settings  (e.g.,  clinics)  (Hilty  et  al,  2013).  These  studies  have  also  generally 
supported  evidence  of  patient  satisfaction  with  TBH  care  (Jenkins-Guamieri,  Pruitt,  Luxton,  & 
Johnson,  2015).  Evaluations  of  the  feasibility,  safety,  and  efficacy  of  TBH  to  the  home  or  any 
setting  should  consider  the  extant  data  that  is  available,  including  future  results  from  trials  that 
are  presently  underway  (e.g.,  Acierno,  Gros,  Morland,  Greene,  Strachan,  Egede,  et  al,  2013). 
Also,  economic  evaluations  of  TBH,  particularly  HBTBH,  are  needed  to  assist  with  decision 
making  regarding  implementation  of  HBTMH  in  the  MHS  (Luxton,  2013).  Economic  analyses 
with  the  present  data  are  planned. 

In  conclusion,  TBH  services  provided  to  the  home  or  other  locations  have  the  potential  to 
address  current  and  future  health  needs  of  military  service  members  and  Veterans,  especially  for 
those  who  live  in  rural  or  underserved  areas.  The  benefits  of  home-based  care  also  extend  to 
family  members  or  other  caregivers  who  may  otherwise  share  the  financial  burden  and 
inconvenience  of  assisting  with  travel  to  health  care  services.  The  value  of  home-based  treatment 
lies  in  its  potential  to  increase  access  to  care  for  a  population  that  has  a  well-documented  history 
of  low  treatment  seeking  behavior  and  other  barriers  to  overcome  (e.g.,  stigma,  frequent 
relocations,  deployment,  and  highly  demanding  work  duty).  The  bottom  line  is  that  home-based 
care  is  a  viable  option,  especially  when  traditional  in-office  care  is  less  practical. 
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Table  1.  Baseline  demographic  characteristics  and  outcome  measures  of  the  trial  study  subjects, 
by  treatment  assignment 


Variable 

n 

In-home 

% 

In- 

n 

-person 

% 

Age 

19-24 

12 

19.35 

10 

16.95 

25-29 

16 

25.81 

12 

20.34 

30-34 

13 

20.97 

11 

18.64 

35-39 

4 

6.45 

9 

15.25 

40-49 

6 

9.68 

8 

13.56 

50-65 

11 

17.74 

9 

15.25 

Sex 

Male 

52 

83.87 

47 

79.66 

Female 

10 

16.13 

12 

20.34 

Race/ethnicity 

White,  non-Hispanic 

44 

70.97 

41 

69.49 

Black,  non-Hispanic 

8 

12.90 

10 

16.95 

Asian,  non-Hispanic 

3 

4.84 

1 

1.69 

Native  American,  non-Hispanic 

1 

1.61 

0 

0.00 

Hispanic,  any  race 

3 

4.84 

7 

11.86 

Other/Unknown 

3 

4.84 

0 

0.00 

Education 

HS 

13 

20.97 

16 

27.12 

Some  college 

32 

51.61 

24 

40.68 

2-year  college 

8 

12.90 

13 

22.03 

4-year  college 

9 

14.52 

6 

10.17 

Years  of  military  service 

1-2  years 

9 

14.52 

14 

23.73 

3-4  years 

14 

22.58 

12 

20.34 

5-8  years 

15 

24.19 

12 

20.34 

9-20  years 

21 

33.87 

19 

32.20 

21-34  years 

3 

4.84 

2 

3.39 

Highest  pay  grade 

E1-E4 

30 

48.39 

26 

44.07 

E5-E9 

29 

46.77 

29 

49.15 

Officer 

3 

4.84 

4 

6.78 

Any  deployment  history 

No 

14 

22.58 

15 

25.42 

Yes 

48 

77.42 

44 

74.58 
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Table  2.  Inclusion  and  Exclusion  Criteria 
Inclusion  Criteria 

(a)  Met  diagnostic  criteria  for  minor  depressive  disorder  or  major  depressive  disorder,  as 
detennined  by  the  SCID-I/P 

(b)  High  speed  internet  access  at  home  (384  kbs  minimum) 

(c)  If  taking  psychoactive  medications,  has  maintained  a  stable  regimen  for  a  minimum  of 
30  days  prior  to  study  entry 

(d)  Infonned  consent  read  and  signed 
Exclusion  Criteria 

(a)  Currently  undergoing  psychotherapy  for  depression 

(b)  b  1 8  or  N65  years  of  age 

(c)  Active  psychotic  symptoms/disorder  as  detennined  by  the  SCID-I/P  (d)  Dysthymic 
disorder  as  determined  by  the  SCID-EP 

(e)  Current  suicidal  ideation  with  intent  or  recent  (within  six  months)  history  of  a  suicide 
attempt 

(f)  History  of  organic  mental  disorder 

(g)  Current  substance  dependence  as  determined  by  the  SCID-EP 
(lifetime  substance  dependence  or  substance  abuse  will  not  be  excluded) 

(h)  History  of  violence  or  poor  impulse  control 

(i)  Significant  ongoing  stressors  that  require  urgent  crisis  intervention 

(j)  Have  a  living  arrangement  that  will  not  permit  the  use  of  a  private  space  to  participate 
in  the  study 
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Table  3.  Intent-to-treat  and  per  protocol  analyses  and  internal  consistency  reliability  estimates  of  the  BHS  and  BDI-II  against  a  non¬ 
inferiority  margin  of  a  standardized  difference  of  0.50. 


In-home  In-Person 


Outcome  and 
time 

n 

M  (SD) 

a 

n 

M  (SD) 

a 

b 

90%  Cl 

B 

90%  Cl 

Intent-to-treat 

BHS 

Pre 

62 

9.00(5.12) 

0.88 

59 

10.37(6.13) 

0.92 

Ref. 

Ref. 

Mid 

48 

7.21  (5.66) 

0.88 

44 

7.95  (6.26) 

0.94 

1.29 

-0.30,2.88 

0.22 

-0.05,  0.50 

Post 

45 

4.89  (4.64) 

0.90 

42 

4.43  (4.94) 

0.92 

2.30 

0.68,3.92 

0.40 

0.12,  0.68 

Follow-up 

42 

5.21  (5.10) 

0.92 

36 

5.53  (5.97) 

0.94 

1.63 

-0.05,3.32 

0.28 

-0.01,  0.58 

os  =  5.77 
ICC  =  0.64 

BDI 

Pre 

62 

27.60  (10.45) 

0.88 

59 

29.71  (11.33) 

0.89 

Ref. 

Ref. 

Mid 

47 

19.40  (11.77) 

0.93 

44 

20.20  (13.09) 

0.93 

1.70 

-1.81,5.22 

0.14 

-0.15,  0.44 

Post 

45 

13.82  (12.02) 

0.94 

42 

11.74  (12.08) 

0.95 

4.23 

0.66,7.81 

0.36 

0.06,  0.66 

Follow-up 

42 

14.76  (12.89) 

0.95 

36 

15.00(12.61) 

0.93 

1.89 

-1.84,5.61 

0.16 

-0.16,  0.48 

os  =  11.78 
ICC  =  0.60  _ 

Per  protocol 

BHS 

Pre 

40 

8.58  (5.38) 

0.90 

42 

11.00  (6.27) 

0.93 

Ref. 

Ref. 

Mid 

40 

7.28  (5.67) 

0.87 

42 

7.55  (6.09) 

0.93 

2.15 

0.43,3.87 

0.39 

0.08,  0.70 

Post 

40 

4.63  (4.42) 

0.89 

42 

4.43  (4.94) 

0.92 

2.62 

0.90,  0.43 

0.47 

0.16,  0.78 

Follow-up 

38 

4.82  (4.99) 

0.92 

36 

5.53  (5.97) 

0.94 

1.90 

0.12,3.68 

0.34 

0.02,  0.67 

Os  =  5-55 
ICC  =  0.63 

BDI 

Pre 

40 

26.65  (11.80) 

0.90 

42 

29.69  (11.74) 

0.90 

Ref. 

Ref. 

Mid 

39 

18.97  (11.89) 

0.93 

42 

19.12  (12.04) 

0.92 

2.74 

-1.08,6.56 

0.22 

-0.09,  0.54 

Post 

40 

13.63  (12.47) 

0.95 

42 

11.74  (12.08) 

0.95 

4.93 

1.12,  8.73 

0.40 

0.09,  0.71 

Follow-up 

38 

14.58  (13.18) 

0.95 

36 

15.00(12.61) 

0.93 

2.64 

-1.29,6.58 

0.22 

-0.11,0.54 

(TS  =  12.23 

ICC  =  0.63 _ 

Note:  BHS  =  Beck  Hopelessness  Scale;  BDI-II  =  Beck  Depression  Inventory  -  II;  b  =  unstandardized  difference  between  in-home  and  in-person  treatment 
groups;  Cl  =  confidence  interval;  B  =  standardized  difference  between  in-home  and  in-person  treatment  groups  using  the  baseline  standard  deviation;  ICC  = 
intraclass  correlation  (time  nested  within  subject). 


HOME-BASED  TELEBEHAVIORAL  HEALTH 


Table  4.  Intent-to-treat  analysis  and  internal  consistency  reliability  estimates  of  secondary  outcomes. 


In-home 

In-person 

Outcome 
and  time 

n 

M  (SD) 

a 

n 

M  (SD) 

a 

b 

95%  Cl 

B 

BAI 

Pre 

62 

15.48  (10.40) 

0.90 

59 

16.85  (12.92) 

0.94 

Ref. 

Ref. 

Mid 

48 

12.21  (9.75) 

0.91 

44 

13.00(10.98) 

0.93 

1.29 

-2.23,  4.82 

0.12 

Post 

45 

9.71  (8.67) 

0.90 

42 

8.31  (9.11) 

0.92 

3.13 

-0.46,  6.74 

0.30 

Follow-up 

Os  =  10.56 

42 

11.10(8.63) 

0.88 

36 

9.75  (8.95) 

0.92 

3.10 

-0.64,  6.84 

0.29 

ICC  =  0.65 
PCL-M 

Pre 

62 

43.15  (13.53) 

0.91 

59 

45.17(14.75) 

0.92 

Ref. 

Ref. 

Mid 

48 

37.60(13.82) 

0.92 

44 

37.41  (16.24) 

0.95 

2.20 

-2.18,6.58 

0.15 

Post 

45 

33.16(14.69) 

0.94 

42 

32.43  (16.58) 

0.96 

2.14 

-2.33,6.61 

0.15 

Follow-up 
Os=  14.67 
ICC  =  0. 72 

42 

35.05  (14.57) 

0.93 

36 

34.39(14.72) 

0.93 

2.26 

-2.39,6.91 

0.15 

IASMHS 

Pre 

62 

35.23  (14.40) 

0.83 

59 

31.47(14.33) 

0.83 

Ref. 

Ref. 

Post 

45 

27.07(14.98) 

0.87 

42 

21.88  (13.75) 

0.85 

1.53 

-3.46,  6.52 

0.10 

Follow-up 

Os  =15.16 
ICC  =  0.68 

42 

28.33  (17.31) 

0.91 

36 

23.69(16.57) 

0.90 

0.24 

-4.97,  5.44 

0.02 

CSQ 

Post 

45 

28.76  (3.41) 

0.91 

42 

29.29  (3.98) 

0.92 

-0.53 

-2.11,-1.05 

-0.14 

Note:  b  =  unstandardized  difference  between  in-home  and  in-person  treatment  groups;  Cl  =  confidence  interval;  B  =  standardized 
difference  between  in-home  and  in-person  treatment  groups  using  the  baseline  standard  deviation;  ICC  =  intraclass  correlation  (time 
nested  within  subject);  BAI  =  Beck  Anxiety  Inventory;  PCL-M  =  PTSD  Checklist  -  Military;  IASMHS  =  Inventory  of  Attitudes 
toward  Seeking  Mental  Health  Service. 
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Table  5.  Estimates  from  a  selection  model  of  dropout  on  the  BHS  and  the  BDI-II 


BHS 

BDI-II 

Model  parameter 

b  (SE) 

b  (SE) 

Intercept 

11.73  (1.72) 

31.77  (3.17) 

In-home 

-1.32  (1.03) 

-2.20(1.98) 

Slope 

-4.26  (0.85) 

-10.93  (1.90) 

In-home 

1.12  (0.49) 

2.11  (1.19) 

Drop  by  mid  assessment3 

In-home 

-0.13  (0.44) 

-0.17(0.44) 

Baseline 

0.02  (0.05) 

-0.01  (0.03) 

Mid 

-0.05  (0.05) 

0.02  (0.04) 

Drop  by  post  assessment3 

In-home 

0.65  (1.02) 

0.78  (0.81) 

Mid 

0.19(0.20) 

0.12(0.11) 

Post 

-0.19(0.58) 

-0.08  (0.11) 

Mean  difference 

b  (90%  Cl)  B  (90%  Cl) 

b  (90%  Cl)  B  (90%  Cl) 

Mid 

1.12  0.20 
(0.31,1.93)  (0.05,0.34) 

2.11  0.19 

(0.15,4.08)  (0.02,0.37) 

Post 

2.24  0.39 

(0.62,3.85)  (0.11,0.68) 

4.23  0.38 

(0.30,8.15)  (0.03,0.73) 

Note:  BHS  =  Beck  Hopelessness  Scale;  BDI-II  =  Beck  Depression  Inventory  -  II;  Cl  =  confidence  interval. 
aModeled  using  a  logistic  link  function  and  a  binomial  error  distribution. 
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Figure  1.  CONSORT  diagram 
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Figure  2.  Intent-to-treat  (ITT)  and  per  protocol  (PP)  assessment  of  non-inferiority  on  the  BHS  at 
each  measurement  occasion  using  a  90%  confidence  interval  (Cl) 
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Figure  3.  Intent-to-treat  (ITT)  and  per  protocol  (PP)  assessment  of  non-inferiority  on  the  BDI  at 
each  measurement  occasion  using  a  90%  confidence  interval  (Cl) 
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Footnotes 


1  A  post  hoc  sample  size  recalculation  based  on  a  power  level  of  0.90  and  a  two-tailed  alpha  of 
0.05  (Greene,  Morland,  Durkalski,  &  Frueh,  2008)  accounted  for  a  difference  between  group 
means  (9)  of  up  to  0.20,  a  margin  of  0.50,  and  an  intra-class  correlation  of  0.50  would  have 
yielded  a  sample  size  of  176  subjects  per  treatment  group.  Since  the  observed  mean  differences 
for  both  outcomes  exceeded  a  small  effect  size  of  0.20,  it  is  likely  that  we  would  still  fail  to 
reject  the  null  hypothesis  of  inferiority  in  excess  of  the  margin. 


Original  Research 

An  Evaluation  of  the  Feasibility  and  Safety  of  a  Home-Based  Telemental  Health 
Treatment  for  Posttraumatic  Stress  in  the  U.S.  Military 


David  D.  Luxton,  PhD,1’2'*  Larry  D.  Pruitt,  PhD,1 
Karen  O'Brien,  PhD 1  and  Gregory  Kramer,  PhD1 

1 National  Center  for  Telehealth  and  Technology,  Joint  Base 
Lewis-McChord,  Washington. 

2 Department  of  Psychiatry  and  Behavioral  Sciences,  University 
of  Washington  School  of  Medicine,  Seattle,  Washington. 
*Current  address:  Naval  Health  Research  Center,  San  Diego, 
California. 

This  study  is  the  initial  part  of  a  clinical  trial  registered  at  ClinicalTrials.gov 
with  clinical  trial  identifier  number  NCT01 599585. 

The  views,  opinions,  and/or  findings  contained  in  this  article  are  those  of 
the  authors  and  should  not  be  construed  as  an  official  Department  of 
Defense  position,  policy,  or  decision  unless  so  designated  by  other  official 
documentation. 


Abstract 

Background:  Although  home-based  telemental  health  options 
have  the  potential  to  greatly  expand  the  range  of  services 
available  to  U.S.  military  service  members,  there  remains  a 
need  to  demonstrate  that  home-based  care  is  technically  fea¬ 
sible,  safe,  and  effective  and  meets  the  military  health  system ’s 
standards  of  care  before  widespread  implementation  can  be 
achieved.  The  purpose  of  this  preliminary  study  was  to  eval¬ 
uate  the  feasibility  and  safety  of  providing  U.S.  military  service 
members  with  a  behavioral  health  treatment  delivered  directly 
to  the  home  using  videoconferencing.  Materials  and  Methods: 
Ten  previously  deployed  soldiers  volunteered  to  complete  eight 
sessions  of  a  novel  behavioral  activation  treatment  for  post- 
traumatic  stress  disorder.  The  primary  clinical  outcomes  as¬ 
sessed  included  symptoms  of  posttraumatic  stress  and 
depression.  Patient  safety  data  and  attitudes  about  seeking 
mental  health  services,  treatment  satisfaction,  treatment  ad¬ 
herence,  and  treatment  compliance  were  also  assessed.  Re¬ 
sults:  Clinically  significant  reductions  in  posttraumatic  stress 
symptom  severity  and  depression  symptoms  were  observed. 
Soldiers  indicated  high  levels  of  satisfaction  with  the  treat¬ 
ment,  and  there  were  no  adverse  events  requiring  activation  of 
emergency  safety  procedures.  Technical  problems  associated 
with  the  network  were  observed  but  successfully  mitigated. 
Conclusions:  The  results  provide  initial  support  for  the  feasi¬ 


bility  and  safety  of  telemental  health  treatments  delivered  by 
videoconferencing  to  the  homes  of  soldiers.  The  optimal  tech¬ 
nical  infrastructure  needs  to  be  determined  to  support  expan¬ 
sion  of  synchronous  videoconferencing  capabilities  to  the 
home.  The  findings  provide  preliminary  evidence  of  the  feasi¬ 
bility,  safety,  and  high  user  satisfaction  with  home-based 
telemental  health  in  the  military  setting. 

Key  words:  telemental  health,  home-based,  behavioral 
activation,  military,  posttraumatic  stress  disorder 

Introduction 

ome-based  telemental  health  (HBTMH)  is  the  pro¬ 
vision  of  mental  healthcare  services  directly  to  the 
homes  of  patients  with  the  use  of  communications 
technologies.  The  U.S.  Veterans  Health  Adminis¬ 
tration  has  successfully  implemented  a  national  home  tele¬ 
health  program  that  included  veterans  with  posttraumatic 
stress  (PTS)  disorder  (PTSD),  depression,  and  chronic  medical 
conditions.1  A  HBTMH  pilot  program2  and  several  clinical 
studies3,4  have  also  demonstrated  the  benefits  of  home-based 
treatments  for  veterans.  The  potential  benefits  of  HBTMH  have 
also  been  recognized  for  some  time  within  the  U.S.  Department 
of  Defense.  For  example,  a  2010  memorandum  from 
the  Chairman  of  the  Joint  Chiefs  of  Staff  advocated  that  the 
military  health  system’s  model  of  care  “...must  deliver  options 
for  mental  health  services  in  the  comfort  and  security  of  the 
Service  member’s  own  home....”5  Service  members  who  are 
living  in  geographically  remote  locations  or  in  areas  that  have  a 
shortage  of  specialty  healthcare  professionals  may  especially 
benefit  from  HBTMH  options.  Moreover,  some  service  members 
may  be  drawn  to  HBTMH  because  of  the  privacy  it  offers  to 
those  who  are  concerned  about  stigma  associated  with  seeking 
mental  health  treatment.  Despite  the  call  for  home-based  care 
within  the  U.S.  Department  of  Defense,  and  the  benefits  it  may 
offer,  the  military  health  system  has  not  established  the  neces¬ 
sary  policies  and  pathways  for  a  HBTMH  model  of  care  to  occur. 

The  U.S.  military  has  unique  network  and  data  security 
requirements  compared  with  other  settings  as  well  as  specific 
protocols  and  procedures  for  care  that  have  not  yet  been  tested 
for  HBTMH.  Although  the  existing  empirical  literature 
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provides  initial  support  and  guidance  for  the  safe  and  effective 
use  of  home  telehealth  services  for  appropriate  populations,6,7 
there  remains  a  need  to  demonstrate  that  home-based  care  is 
technically  feasible,  safe,  and  effective  and  meets  the  military 
health  system’s  standards  of  care  before  widespread  imple¬ 
mentation  can  be  achieved.  The  purpose  of  the  present  study  was 
therefore  to  evaluate  the  feasibility  of  providing  a  behavioral 
activation  (BA)  treatment  for  PTSD8  delivered  via  synchronous 
(two-way)  videoconferencing  to  the  homes  of  U.S.  military 
service  members.  BA  is  a  well-established  treatment  for  de¬ 
pression  that  counters  patterns  of  avoidance  and  withdrawal 
with  a  pattern  of  engagement  in  valued  activities  through  ac¬ 
tivity  planning.9  Given  that  avoidance  and  withdrawal  processes 
also  serve  to  maintain  the  symptoms  of  PTSD,  BA  has  been 
evaluated  as  a  treatment  for  PTSD.8'10-12  Although  previous 
studies  have  examined  military  veterans,  our  study  is  the  first  to 
evaluate  BA  for  PTS  among  active  duty  military  members. 

Materials  and  Methods 

This  study  is  an  initial  part  of  a  multisite  clinical  trial  that  is 
comparing  the  effectiveness  of  BA  for  depression  delivered  in¬ 
office  versus  in  the  home.13  Although  the  randomized  clinical 
trial  provides  an  opportunity  to  test  the  effectiveness  of  home- 
based  BA  for  service  members  and  veterans  with  depression, 
this  evaluation  allowed  us  to  assess  the  feasibility  of  the 
technology,  to  examine  safety  management  procedures,  and 
to  evaluate  a  promising  treatment  for  PTSD.  We  predicted  that 
the  BA  intervention  would  result  in  a  reduction  in  PTS  and 
depression  symptoms.  We  also  included  measures  of  anxiety 
and  sleep  quality  as  exploratory  outcomes.  The  procedures  of 
this  study  adhere  to  the  principles  and  recommendations  of 
the  World  Medical  Association  and  the  Declaration  of  Helsinki 
Ethical  Principles  for  Medical  Research  Involving  Human 
Subjects.  This  study  was  approved  by  the  Madigan  Army 
Medical  Center  Institutional  Review  Board  and  the  U.S.  Army 
Medical  Research  and  Materiel  Command’s  Human  Research 
Protection  Office. 

PARTICIPANTS 

The  sample  consisted  of  10  active  duty  members  of  the  U.S. 
Army.  All  participants  were  referred  to  the  study  from 
medical  and  behavioral  health  clinics  at  a  large  Army  med¬ 
ical  treatment  facility.  The  study  inclusion  and  exclusion 
criteria  were  determined  by  initial  screening  interview.  To  be 
eligible  for  the  study,  participants  had  to  endorse  experi¬ 
encing  at  least  one  criterion  A  stressor  and  have  a  score  of  45 
or  higher  on  the  Clinician  Administered  PTSD  Scale 
(CAPS).14  The  cutoff  score  of  45  has  been  used  in  other 
studies.15  Participants  taking  any  psychoactive  medications 


had  to  have  maintained  a  stable  regimen  for  a  minimum  of 
30  days  prior  to  study  entry. 

PROCEDURES 

As  part  of  the  informed  consent  process,  participants  were 
provided  with  detailed  information  about  how  their  identity 
and  private  health  information  would  be  protected,  the  limits 
of  confidentiality,  and  the  record  keeping  system  used  in  the 
study.  This  included  an  overview  of  the  telehealth  equipment 
and  instructions  pertaining  to  setting  up  the  treatment  envi¬ 
ronment  in  a  private  area  free  from  distractions.  All  partici¬ 
pants  completed  a  release  of  information  form  so  that  a 
contact  person,  of  their  choice,  could  assist  in  case  of  clinical 
emergency.  The  requirements  and  processes  for  engaging  with 
third  parties  were  disclosed  and  discussed  during  the  informed 
consent  process. 

Study  personnel.  Treatment  providers  included  five  clini¬ 
cians  (four  licensed  psychologists  and  one  postdoctoral  fel¬ 
low).  The  postdoctoral  fellow  was  supervised  on  a  weekly 
basis.  All  study  providers  participated  in  initial  and  annual  BA 
training  workshops. 

Clinical  assessment.  PTS  symptom  severity  assessments  were 
completed  by  two  independent,  doctoral-level  outcomes  as¬ 
sessors.  These  assessors  were  trained  in  the  administration  of 
the  CAPS  and  possessed  prior  experience  with  this  measure. 
The  CAPS,  along  with  the  self-report  battery,  was  completed 
at  the  baseline,  midtreatment,  posttreatment,  and  3-month 
posttreatment  assessments.  All  assessments  were  video- 
recorded  and  reviewed  regularly  by  a  supervisory  psycholo¬ 
gist.  Feedback  was  provided  on  an  as  needed  basis  to  improve 
compliance  with  CAPS  administration  rules. 

Treatment  protocol.  The  intervention  consisted  of  eight  ses¬ 
sions  of  BA  for  PTS.  The  treatment  protocol  is  adapted  from  a 
BA  treatment  manual11  that  has  been  expanded  into  an  early 
intervention  for  PTSD  and  depression.8  The  protocol  places  a 
strong  emphasis  on  an  outside-in  approach  to  behavior 
change  whereby  values-consistent  activities  are  identified, 
planned,  and  tracked.  The  treatment  is  guided  by  behavioral 
theory,  with  functional  behavioral  analysis  being  a  primary 
treatment  component.16 

The  primary  tasks  during  the  first  two  treatment  sessions  are 
(a)  to  provide  psychoeducation  about  PTSD  and  the  rationale 
for  treatment,  (b)  to  identify  values,  priorities,  and  treatment 
goals  and  to  translate  those  into  scheduled  activities,  and  (c)  to 
establish  a  pattern  of  daily  monitoring  and  planning  for  ac¬ 
tivities.  The  goal  of  the  remaining  sessions  is  to  support  the 
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ongoing  implementation  of  BA  strategies.  During  sessions  3- 
8,  the  treatment  provider  conducts  functional  analyses  of 
avoidance  behaviors  that  prevent  participants  from  engaging 
in  scheduled  activities  and  reinforce  progress  towards  goals. 
During  sessions  7  and  8,  the  provider  also  discusses  relapse 
prevention  and  encourages  participants  to  use  BA  principles 
if/when  symptoms  return. 

Telehealth  procedures.  All  participants  were  issued  a  Dell 
(Round  Rock,  TX)  M6500  laptop  computer  and  a  Tandberg 
(Oslo,  Norway)  Precision  High  Definition  Webcam.  Participants 
were  also  provided  with  a  username  and  password  for  access  to 
preloaded  Jabber  Video  software  (Cisco  Systems,  San  Jose,  CA). 
This  software  was  selected  because  its  level  of  security  and 
encryption  is  approved  for  use  by  the  U.S.  Army.  Prior  to  the 
first  treatment  session,  a  treatment  station  set-up  appointment 
was  scheduled  between  participant  and  treatment  provider  to 
familiarize  participants  with  the  equipment  and  the  Jabber 
Video  software  and  to  test  the  network  connection.  Participants 
were  required  to  use  their  home  Wi-Fi  or  cable  Internet  con¬ 
nections  to  log-in  for  treatment  sessions.  Participants  were 
instructed  to  initiate  the  Jabber  Video  connection  with  their 
assigned  treatment  provider  at  scheduled  appointment  times. 

Some  modifications  to  the  original  BA  protocol  were  nec¬ 
essary  in  order  to  deliver  the  treatment  remotely  via  telehealth 
technology.  A  treatment  session  checklist12  was  administered 
at  the  beginning  of  each  session  for  the  purpose  of  reminding 
study  clinicians  of  procedures  and  for  documenting  patient 
safety  and  technical  issues.  Several  modifications  were  also 
required  for  sharing  homework  and  study  handouts  (BA 
worksheets,  self-report  questionnaires,  etc.),  such  as  use  of 
screen  shots  of  homework  and  handouts  and  holding  hand¬ 
outs  up  to  the  camera. 

Measures.  The  following  measures  were  assessed: 

•  Demographic  questionnaire.  Participants  provided  de¬ 
mographic  information  including  occupation/work 
status/income/living  situation,  branch  of  service/highest 
rank,  pain  rating  (0-10),  and  medications. 

•  CAPS.  The  CAPS14  is  a  structured  interview  that  assesses 
all  DSM-IV  PTSD  criteria  in  terms  of  frequency  and  in¬ 
tensity.  The  CAPS  Current  and  Lifetime  Version,  which 
measures  a  1 -month  symptom  duration,  was  used  for  the 
baseline  and  follow-up  assessments.  The  CAPS  One  Week 
Version,  which  measures  symptoms  over  the  past  week, 
was  used  to  assess  participants  after  treatment  sessions  4 
and  8.  PTSD  severity,  measured  by  the  CAPS  (total  score), 
served  as  the  primary  PTSD  outcome. 


•  PTSD  Checklist  Military  Version.  The  PTSD  Checklist17  is 
a  self-report  measure  that  evaluates  all  17  DSM-IV  PTSD 
symptoms  across  the  three  primary  symptom  clusters 
using  a  5-point  Likert  scale.  Internal  consistency  for  the 
total  score  is  high  (0.97),  as  are  reliability  estimates 
(0.96).  The  PTSD  Checklist  Military  Version  (PCL-M)  is 
used  here.  A  total  score  of  50  typically  serves  as  the 
threshold  for  identifying  probable  PTSD  among  those 
reporting  military-related  trauma(s). 

•  Beck  Depression  Inventory-II.  The  Beck  Depression 
Inventory-II18  (BD1-II)  is  the  most  commonly  used  self- 
report  measure  of  clinical  depression  severity.  It  consists 
of  21  items  that  are  rated  on  a  4-point  scale  and  that 
yield  a  range  of  scores  from  0  to  63. 

•  Beck  Anxiety  Inventory.  The  Beck  Anxiety  Inventory19 
(BAI)  is  a  self-report  measure  consisting  of  21  items 
designed  to  discriminate  anxiety  from  depression.  It  has 
high  internal  consistency  (0.92)  and  1-week  test-retest 
reliability  (0.75)  and  discriminates  anxious  from  non- 
anxious  diagnostic  groups.20 

•  Pittsburgh  Sleep  Quality  Index.  The  Pittsburgh  Sleep 
Quality  Index  (PSQI)21  is  a  10-item  measure  of  sleep 
quality.  This  measure  assesses  both  the  quality  and  quantity 
of  an  individual’s  sleep  pattern  over  a  1 -month  period. 
Internal  consistency  for  this  measure  has  been  found  to  be 
0.80,  with  a  reliability  coefficient  of  0.83  and  test-retest 
reliability  of  0.87.22 

•  Safety  measures.  Safety  data  collected  included  any  ad¬ 
verse  events,  psychiatric  hospitalizations,  suicides  and 
nonfatal  suicide-related  behaviors,  number  of  times  the 
patient  support  person  was  utilized  during  treatment, 
treatment  adherence,  and  frequency  of  requests  for  patient 
or  therapist  technical  support.  Safety-related  data  were 
recorded  after  each  treatment  session  on  the  Treatment 
Session  Checklist.  We  also  followed  the  suicide  assessment 
and  risk  management  Standard  Operating  Procedure  (SOP) 
used  at  Madigan  Army  Medical  Center  to  assess  and 
document  suicide  risk.  The  SOP  requires  clinicians  to  as¬ 
sess  and  document  current  ideation,  presence  of  a  plan, 
suicidal  intent,  history  of  previous  attempts,  and  degree  of 
impulsivity.  Risk  correlates  (e.g.,  recent  loss,  financial 
problems),  preparatory  behavior  (e.g.,  available  means), 
and  other  risk  factors  (e.g.,  substance  dependence)  are  also 
assessed  and  documented.  The  SOP  was  administered  at 
the  baseline  assessment  and  the  first  treatment  session  and 
re-administered  at  each  subsequent  session  if  a  patient 
endorsed  current  elevated  risk  per  the  SOP. 

•  Treatment  Session  Checklist.  The  Treatment  Session 
Checklist13  is  designed  to  collect  information  for  the 
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evaluation  of  clinical  telehealth  sessions.  It  is  used  to 
document  safety  information  including  current  suicidal 
ideation,  homicidal  ideation,  and  other  signs  of  risk 
(including  the  visual  presence  of  a  weapon  at  the  pa¬ 
tient’s  location).  Clinical  factors  such  as  indicators  of 
intoxication,  disorientation,  and  severe  emotion  dysre- 
gulation  are  also  included,  as  are  questions  related  to  the 
in-home  environment,  such  as  “Is  anyone  else  at  home 
today?”  and  “Do  you  feel  that  your  environment  is  safe 
and  private?”  This  checklist  is  also  used  to  document 
telehealth  equipment  and  connectivity  status,  adequate 
lighting,  and  any  disruptions  to  session. 

•  Client  Satisfaction  Questionnaire.  The  Client  Satisfac¬ 
tion  Questionnaire23  is  an  eight-item  self-report  mea¬ 
sure  of  general  satisfaction  with  psychotherapeutic 
treatment.  (The  instrument  is  reproduced  with  permis¬ 
sion  of  C.  Clifford  Attkisson.)  Participants  are  asked  to 
rate  satisfaction  on  a  4-point  scale,  with  a  possible 
range  of  8  to  32,  with  higher  scores  indicating  greater 
satisfaction.  Internal  consistency  and  construct  validity 
have  been  established,  and  the  measure  is  widely  used 
in  research.24 

Results 

All  participants  were  men,  between  the  ages  of  21  and  45 
years  with  a  mean  age  of  3 1.8  (standard  deviation  [SD]  =  7.44) 
years.  All  were  enlisted  members 
of  the  U.S.  Army  with  an  average 
length  of  military  service  of 
9.3  (SD  =  5.2 1)  years.  Five  of  the 
10  participants  reported  having 
some  college  education,  and  9  of 
the  participants  reported  that  they 
were  currently  married.  Seven 
of  the  10  participants  resided  in 
private  housing  off  of  the  military 
installation. 

Seven  of  the  10  participants  had 
deployed  to  Iraq  at  least  once  in 
support  of  Operation  Iraqi  Free¬ 
dom,  and  6  had  been  deployed  to 
Afghanistan  in  support  of  Opera¬ 
tion  Enduring  Freedom.  Two  par¬ 
ticipants  had  also  experienced 
other  deployments.  The  number  of 
deployments  that  any  single  par¬ 
ticipant  reported  ranged  from  one 
to  four.  All  of  the  index  traumas 


assessed  on  the  CAPS  were  combat  related  and  occurred  during 
Operation  Enduring  Freedom/Operation  Iraqi  Freedom  deploy¬ 
ments.  All  of  these  traumatic  events  met  DSM-IV  criterion  A  for 
PTSD.  On  average,  these  traumas  had  occurred  6  (SD  =  3.33) 
years  prior  to  the  patient  presenting  for  treatment.  The  span  of 
time  since  trauma  exposure  ranged  from  2  to  1 1  years. 

CLINICAL  OUTCOMES 

We  examined  clinical  treatment  outcomes  based  on  similar 
procedures  used  in  a  pilot  study  of  BA  for  PTSD  that  was 
conducted  with  veterans.11  Table  1  shows  the  results  of 
paired-sample  t  tests  (two-tailed)  of  clinical  outcome  mea¬ 
sures  and  individual  responses  to  the  treatment.  We  calculated 
Hedge’s  g  to  represent  effect  size  and  used  Cohen’s  definition25 
to  interpret  them.  The  criteria  we  used  for  reliable  change  was 
based  off  of  previous  research11,26  and  was  as  follows: 
CAPS,  ±  9 ;  PCL-M,  ±  5 ;  BDI-II,  +  5 ;  BAI,  ±  8 ;  and  PSQI,  +  2. 

As  shown  in  Figure  1,  there  was  a  trend  of  decreased 
symptom  levels  from  pre-  to  posttreatment  for  all  clinical 
measures.  There  was  a  statistically  reliable  decrease  in  PTS 
severity  and  symptoms  as  measured  by  the  CAPS  and  the  PCL- 
M,  with  five  participants  showing  improvement  on  the  CAPS 
and  seven  on  the  PCL.  There  was  a  statistically  reliable  reduc¬ 
tion  in  BDI-II  scores,  with  6  patients  meeting  criteria  for  clinical 
improvement.  There  was  no  reliable  change  in  mean  scores  for 
the  BAI;  however,  five  participants  showed  improvement,  and 


Table  1.  Treatment  Clinical  Outcomes 

MEASURE 

MEAN  (SD) 

BASELINE  POSTTREATMENT 

t  (df) 

HEDGE’S  g 

RC 

CAPS 

82.30  (12.37) 

65.11  (20.74) 

3.29  (8)a 

0.95 

5  improved 

0  deteriorated 

PCL-M 

59.20  (10.40) 

53.22  (14.68) 

2.53  (8)a 

0.44 

7  improved 

1  deteriorated 

BDI-II 

29.50  (10.32) 

20.67  (12.00) 

2.95  (8)a 

0.75 

6  improved 

0  deteriorated 

BAI 

23.20  (11.66) 

19.22  (10.53) 

2.15  (8) 

0.34 

5  improved 

1  deteriorated 

PSQI 

16.20  (3.05) 

15.33  (3.35) 

1.71  (8) 

0.41 

3  improved 

1  deteriorated 

ap<0.05. 

BAI,  Beck  Anxiety  Inventory;  BDI-II,  Beck  Depression  Inventory-ll;  CAPS,  Clinician  Administered  Posttraumatic  Stress 
Disorder  Scale;  PCL-M,  PTSD  Checklist  Military  Version;  PSQI,  Pittsburgh  Sleep  Quality  Index;  RC,  reliable  change;  SD, 
standard  deviation. 
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Fig.  1.  Means  of  clinical  outcomes  measures  at  each  time  point.  BAI,  Beck  Anxiety  Inventory;  BDI, 
Beck  Depression  Inventory;  BHS,  Beck  Hopelessness  Scale;  CAPS,  Clinician  Administered  Post- 
traumatic  Stress  Disorder  Scale;  PCL-M,  Posttraumatic  Stress  Disorder  Checklist  Military  Version; 
PSQI,  Pittsburgh  Sleep  Quality  Index. 


one  deteriorated.  There  was  also  not  a  statistically  reliable 
change  in  the  mean  PSQI  scores  for  the  sample. 

TREATMENT  ADHERENCE  AND  SATISFACTION 

Two  of  the  10  participants  did  not  complete  all  eight 
treatment  sessions;  both  withdrew  from  treatment  following 
session  5.  These  participants  reported  that  despite  noticing 
that  treatment  had  led  to  improvements  in  their  quality  of  life 
and  PTS  symptoms,  participating  in  the  treatment  required  too 
much  time  away  from  their  Army  duties.  One  of  these  par¬ 
ticipants  had  experienced  frequent  technical  difficulties  that 
may  have  been  a  contributing  factor  to  his  decision  to  with¬ 
draw  from  treatment.  Treatment  completers  indicated  high 
overall  satisfaction  with  the  treatment  on  the  CSQ-8  (mean  = 
25.86,  SD  =  4.74). 

TECHNICAL  FEASIBILITY 

As  shown  in  Table  2,  the  most  frequent  technical  issue  was 
difficulty  establishing  a  connection  to  the  videoconferencing 
server.  This  problem  was  typically  resolved  with  additional 
sign-in  attempts.  The  average  length  of  these  disruptions  was 
less  than  6  min.  More  serious  connection  difficulties  developed 
during  an  Army-wide  network  security  upgrade  that  caused  the 
Internet  protocol  addresses  associated  with  some  of  the  laptops 
to  be  blocked  from  accessing  the  video  teleconferencing  soft¬ 


ware’s  network.  This  occurred  over 
a  2.5-month  period  during  the 
study  and  necessitated  multiple 
treatment  sessions  to  be  completed 
via  telephone.  One  participant 
who  completed  the  entire  treat¬ 
ment  protocol  experienced  six 
treatment  sessions  conducted  by 
telephone.  The  second  case  com¬ 
pleted  five  treatment  sessions, 
three  of  which  occurred  over  the 
telephone.  This  participant  with¬ 
drew  from  the  study  after  session 
5.  Although  technical  issues  with 
initiating  and  maintaining  a  vid¬ 
eoconferencing  connection  were 
more  frequent  than  expected,  they 
were  managed  effectively  by  study 
clinicians  with  simple  trouble¬ 
shooting  steps  and  use  of  alter¬ 
native  contact  methods  per  the 
study’s  protocol. 

SAFETY  OUTCOMES 

There  were  no  adverse  events  during  the  study  or  inci¬ 
dences  that  necessitated  activation  of  our  emergency  pro¬ 
tocol.  At  the  baseline  assessment,  one  participant  endorsed 
thoughts  of  suicide  but  reported  no  desire  or  intent  to  act  on 
those  thoughts  and  was  therefore  eligible  for  participation. 
At  the  midpoint  assessment  this  participant  continued  to 
endorse  thoughts  of  suicide.  However,  by  the  end  of  treat¬ 
ment  those  thoughts  were  no  longer  present,  and  suicidal 
ideation  remained  absent  throughout  the  follow-up  period. 
Two  other  patients  (who  did  not  indicate  suicidal  thoughts  at 
baseline)  reported  single-occurrence  endorsements  of  sui¬ 
cidal  thoughts  midway  through  treatment  but  did  not  report 
any  plan  or  intent  to  act  on  those  thoughts.  Both  of  these 
patients  no  longer  reported  suicidal  thoughts  at  posttreat¬ 
ment  or  follow-up  assessments.  None  of  the  participants 
expressed  any  specific  desire  or  intent  to  harm  others,  and 
there  were  no  incidences  that  required  notification  of  a  pa¬ 
tient’s  emergency  contact  person  or  emergency  services. 
There  was  never  a  time  when  a  patient  deliberately  termi¬ 
nated  the  video  teleconferencing  connection  during  a 
treatment  session. 

Discussion 

This  study  is  the  first  to  examine  BA  for  the  treatment  of  PTS 
symptoms  among  active-duty  U.S.  military  personnel  and  the 
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Table  2.  Technical  Difficulties  Occurring  Across  All  73 
In-Home  Telehealth  Sessions 

COUNT  (0/0) 

MEAN  (SD) 

Technical  issue 

Unable  to  immediately  establish  a  VTC 
connection 

31  (42.5) 

Time  (min)  to  establish  a  connection 

5.99  (4.27) 

Was  the  disruption  severe  enough  to 
warrant  phone  contact? 

34  (46.6) 

Patient  was  unable  to  be  contacted  by 
phone  to  follow-up 

0  (0) 

Problem  source  for  the  31  sessions  where  problems  establishing  a  connection 
occurred 

Problematic  Internet  connection 

3  (9.6) 

Software  problems 

1  (3.2) 

Hardware  problems 

3  (9.6) 

Server  problems 

17  (54.8) 

Other/unknown 

7  (22.6) 

VTC  connection  lost  midsession  due  to 
technical  issue 

10  (13.7) 

Time  (min)  to  re-establish  a  connection 

4.71  (4.50) 

Instances  of  being  unable  to  re-establish 
a  VTC  connection 

3  (30) 

Problem  source  for  the  10  sessions  where  problems  maintaining  a  connection 
occurred 

Problematic  Internet  connection 

4  (40) 

Software  problems 

1  (10) 

Hardware  problems 

3  (30) 

Participant  purposely  terminated  contact 

0  (0) 

Other/unknown 

2  (20) 

SD,  standard  deviation;  VTC,  video  teleconferencing. 


first,  to  our  knowledge,  to  assess  a  home-based  synchronous 
telemental  health  intervention  in  the  U.S.  military.  The  overall 
results  provide  initial  support  for  the  feasibility  of  HBTMH 
treatments  in  the  military  setting.  The  results  also  showed 
positive  treatment  effects  of  this  novel  intervention  on  symp¬ 
toms  of  both  PTS  and  posttraumatic  depression  that  are  con¬ 
sistent  with  other  recent  studies.8,10-12  Although  the  results  did 
not  show  statistically  reliable  overall  reduction  with  the  sleep 
quality  measure  (PSQI),  this  is  not  surprising  given  the  small 
sample  size  of  our  study.  The  results  also  suggest  high  levels  of 
treatment  satisfaction  with  HBTMH  and  the  absence  of  any 
safety  events  provide  additional  data  that  mental  healthcare 


can  be  delivered  safely  to  service  members  in  their  homes  when 
using  workable  safety  standards  and  planning. 

Although  we  did  experience  several  temporary  technical  is¬ 
sues  that  caused  some  inconveniences  for  both  patients  and  our 
study  care  providers,  these  issues  did  not  appear  to  be  detri¬ 
mental  to  the  treatment  process.  The  technological  aspects  of 
HBTMH  were  manageable,  and  disruptions  (primarily  caused  by 
the  unanticipated  network  security  upgrade)  were  usually  cor¬ 
rected  within  several  minutes  of  a  problem.  Our  use  of  U.S. 
Army-approved  and  standardized  laptops,  software,  and  cam¬ 
eras  helped  assure  technical  compliance  and  control  of  poten¬ 
tial  hardware  technical  issues;  however,  this  is  also  a  limitation 
of  the  study.  An  ideal  capability  would  be  to  use  a  network 
infrastructure  that  meets  U.S.  Department  of  Defense  network 
security  requirements  but  that  also  allows  for  the  use  of  pri¬ 
vately  owned  end-user  equipment  (i.e.,  personal  computers, 
Webcams,  mobile  devices,  etc.).  Ultimately,  the  capability  to  use 
readily  available  privately  owned  equipment  would  be  more 
convenient  and  economical  for  broad  implementation.  The 
optimal  infrastructure  for  supporting  enterprise-wide  HBTMH 
videoconferencing  capabilities  in  this  setting  needs  to  be  de¬ 
termined. 

In  conclusion,  HBTMH  has  the  potential  to  greatly  expand 
the  range  of  services  available  to  U.S.  military  members, 
veterans,  and  the  general  population.  Although  the  current 
study  is  limited  by  its  small  sample  size  and  lack  of  a  control 
group,  the  findings  support  the  notion  that  it  is  possible  to 
deliver  a  similar  quality  and  standard  of  care  (i.e.,  an  estab¬ 
lished,  evidence-based  treatment)  to  the  home  as  in  the  clinic 
in  the  military  setting.  This  study  can  serve  as  a  model  to 
investigate  and  implement  other  forms  of  home-based 
healthcare,  and  it  provides  decision  makers  with  necessary 
preliminary  data  to  make  decisions  regarding  the  expansion 
of  HBTMH  options  for  the  U.S.  military  community. 
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Abstract 

Background:  Although  there  is  growing  empirical  support 
for  the  clinical  efficacy  of  telemental  health  (TMH)  treat¬ 
ments,  questions  remain  about  how  patient  perceptions  of 
the  TMH  treatment  process  may  compare  with  those  of 
traditional  in-person  psychotherapy  treatments.  Materials 
and  Methods:  Through  a  systematic  review,  we  specifically 
examine  measures  of  patient  treatment  satisfaction  and 
therapeutic  alliance  in  studies  that  included  direct  com¬ 
parisons  of  video  teleconferencing  or  telephone-based  psy¬ 
chotherapeutic  TMH  treatments  with  in-person  treatment 
delivery.  We  performed  a  comprehensive  search  of  the  Psy- 
chlNFO  and  MEDLINE  databases  for  articles  published  in  the 
last  10  years  (2004-2014)  on  TMH  treatments  that  included 
in-person  comparison  groups,  yielding  552  initial  results 
with  14  studies  meeting  our  full  inclusion  criteria.  Results: 
The  findings  generally  show  comparable  treatment  satis¬ 
faction  as  well  as  similar  ratings  of  therapeutic  alliance. 
Some  results  suggested  the  potential  for  decreased  patient 
comfort  with  aspects  of  group  treatment  delivered  via  TMH. 
Conclusions:  We  discuss  implications  for  providing  psy¬ 
chotherapeutic  treatments  via  TMH  and  review  practice 
recommendations  for  assuring  and  enhancing  satisfaction 
with  TMH  services. 

Key  words:  telemedicine,  telemental  health,  telepsychology, 
video  teleconferencing,  treatment  satisfaction,  therapeutic 
alliance,  literature  review 


Introduction 

elemental  health  (TMH)  refers  to  “the  provision  of 
mental  health  and  substance  abuse  services  from  a 
distance.” 1,p'7  Telephone  and  video  teleconferencing 
(VTC)  are  common  synchronous  telecommunication 
technologies  that  are  used  for  providing  direct  psychothera¬ 
peutic  treatments  to  patients.  Research  evaluating  TMH 
treatments  thus  far  has  supported  their  efficacy  in  addressing 
a  broad  range  of  mental  health  concerns  in  many  different 
patient  populations.2'3  Psychotherapeutic  interventions  (e.g., 
behavior  change  through  collaborative  dialogue  in  a  thera¬ 
peutic  relationship)  delivered  via  TMH  have  demonstrated 
comparable  efficacy  to  in-person  treatment  in  rigorously 
designed  randomized  controlled  trials,  including  treatments 
for  posttraumatic  stress  disorder,4’5  depression,6  and  anger 
management.7  Psychiatric  services  (e.g.,  medication  manage¬ 
ment,  symptom  monitoring,  and  reducing  side  effects) 
delivered  through  TMH  have  also  been  shown  to  have  com¬ 
parable  efficacy  to  in-person  psychiatric  care. 8-10  Moreover, 
TMH  treatments  have  been  successfully  delivered  to  a  wide 
range  of  patient  populations,  including  children  and  ado¬ 
lescents,11  adults,12  older  adults,13  military  veterans,14  and 
ethnically  diverse  populations.15 

Although  there  is  mounting  empirical  support  for  the 
clinical  efficacy  of  treatments  delivered  via  TMH,  questions 
remain  in  the  literature  about  patient  perceptions  of  TMH 
services,  especially  regarding  patient  satisfaction  with  remote 
treatment  and  potential  differences  in  therapeutic  alliance  and 
rapport  compared  with  traditional  in-office  treatment  deliv¬ 
ery.2'16  Patient  satisfaction  with  and  perceptions  of  the  ther¬ 
apeutic  relationship  are  highly  relevant  for  the  adoption  and 
acceptance  of  TMH  services  among  both  TMH  providers  and 
patients.  The  American  Telemedicine  Association  Telemental 
Health  Special  Interest  Group  defined  patient  satisfaction 
as  the  “patient’s  subjective  satisfaction  and  experience  with 
the  TMH  service  provided.”17'11'285  Patient  satisfaction  with 
treatment  may  be  an  important  variable  in  evaluating  inter¬ 
vention  programs,18  and  therapeutic  alliance  has  been  shown 
to  be  a  key  factor  related  to  psychotherapy  outcomes.19 
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Previous  reviews  of  TMH  treatments  in  general  (including 
psychotherapeutic  intervention,  telepsychiatiy,  etc.)  have 
generally  found  high  levels  of  patient  satisfaction  ratings.3,20 
Research  focused  on  telepsychiatry  interventions  has  specif¬ 
ically  found  comparable  levels  of  patient  satisfaction  between 
in-person  and  TMH  delivery  of  services.9,10  However,  these 
results  may  not  generalize  well  to  other  types  of  treatment,  as 
psychiatry  is  most  often  concerned  with  medication  man¬ 
agement  during  brief,  intermittent  meetings,  whereas  psy¬ 
chotherapeutic  treatment  relies  on  a  therapeutic  relationship 
and  detailed  discussion  of  patient  concerns  and  intervention 
strategies  that  develop  over  multiple  sessions.  Therefore,  as¬ 
pects  of  and  expectations  for  patient  satisfaction  may  differ 
significantly  between  these  two  types  of  TMH  treatment,  and  it 
may  be  helpful  to  consider  them  separately.  Although  existing 
research  has  generally  found  high  levels  of  satisfaction  with 
TMH  psychotherapeutic  interventions,2,21  no  reviews  to  date 
have  investigated  patient  satisfaction  with  psychotherapeutic 
treatment  compared  with  in-person  delivery.  This  is  an  im¬ 
portant  contribution  to  the  literature  given  that  “we  do  not 
know  whether  patient  satisfaction  with  tele-mental  health 
would  remain  as  high  in  the  presence  of  alternative  mental 
health  services.”20,11'328  In  addition,  there  have  been  some 
indications  that  satisfaction  with  TMH  may  be  reduced,  lar¬ 
gely  due  to  the  potential  for  technical  difficulties  in  TMH 
equipment  and  poor  connection  quality  (e.g.,  low  Internet 
bandwidth).2,20 

Therapeutic  alliance  is  another  important  factor  that  has 
been  shown  to  be  associated  with  psychotherapy  outcomes.19 
Therapeutic  alliance,  also  called  working  alliance,  was  defined 
by  Bordin22  as  consisting  of  three  aspects:  agreement  and 
collaboration  on  goals  for  therapy  work,  the  tasks  patients  are 
engaged  in,  and  the  interpersonal  bond  between  patient  and 
therapist  characterized  by  mutual  trust  and  attachment.  Low 
satisfaction  and  weaker  therapeutic  relationships  in  remote 
TMH  treatment  may  deter  people  from  engaging  in  treatment. 
For  example,  frustration  with  setting  up  or  using  the  tech¬ 
nology  and  concern  about  privacy  may  discourage  patients 
from  seeking  out  or  engaging  in  TMH  treatment.  Providers 
and  their  choices  about  treatment  modalities  used  may  also  be 
impacted  by  their  own  attitudes  toward  and  perceptions  of 
TMH.23  For  example,  in  a  study  that  surveyed  mental  health 
professionals,  Perle  et  al.24  found  lower  general  levels  of  ac¬ 
ceptance  toward  technology-mediated  interventions  com¬ 
pared  with  face-to-face  services,  which  also  varied  widely  by 
type  of  psychiatric  diagnosis.  Some  researchers  have  begun  to 
address  these  issues  more  directly,  with  some  emerging  sup¬ 
port  for  comparable  patient  ratings  of  working  alliance  for  in- 
person  and  e-therapy  (text-based)  treatment.25  Similarly, 


Stiles-Shields  et  al.26  reported  comparable  working  alliance 
scale  scores  from  both  clients  and  providers  in  the  face-to- 
face  and  telephone  cognitive  behavioral  therapy  (CBT)  con¬ 
ditions  in  a  sample  of  individuals  struggling  with  depression. 

Although  preliminary  research  has  begun  to  illuminate 
these  central  questions  about  patient  perceptions  of  TMH 
treatments,  more  information  is  needed  to  fully  evaluate 
TMH-based  treatments  and  to  also  identify  best  practices  that 
assure  optimal  delivery  of  care.  The  literature  has  echoed  these 
calls  for  direct  comparison  between  satisfaction  levels  for 
psychotherapeutic  TMH  treatments  and  face-to-face  condi¬ 
tions,  as  well  as  the  establishment  of  standard  measures  of 
client  satisfaction  for  research  in  this  area.2,23  Thus,  we  sought 
to  conduct  a  systematic  review  of  TMH  research  literature 
to  evaluate  studies  that  directly  compare  psychotherapeutic 
TMH  treatment  with  equivalent  in-person  services  to  syn¬ 
thesize  findings  regarding  patient  satisfaction  and  therapeu¬ 
tic  alliance.  We  also  review  the  factors  reported  to  influence 
patient  satisfaction  with  TMH-based  treatment  services  and 
discuss  best  practices  for  assuring  and  enhancing  satisfaction 
with  TMH  services. 

Materials  and  Methods 

LITERATURE  SEARCH 

We  conducted  a  comprehensive  review  of  published  re¬ 
search  articles  in  the  telemedicine  and  TMH  literatures  by 
performing  a  systematic  search  of  the  PsycINFO  and  Ovid 
MEDLINE  databases.  We  followed  a  systematic  strategy  of 
combining  the  search  terms  “telemedicine,”  “telemental  health,” 
“telehealth,”  “teleconferencing,”  or  “videoconferencing”  with 
every  one  of  the  following  terms  in  combinations  using 
Boolean  logic  operators  (e.g.,  AND,  OR):  "mental  health,” 
“psychotherapy,”  “therapist  attitudes,”  “client  attitudes,”  “cli¬ 
ent  satisfaction,”  and  “treatment.”  Restrictions  for  all  searches 
included  the  following:  English  language,  publication  date 
range  between  2004  and  the  third  week  of  February  2014, 
empirical  research  involving  adults,  and  publication  in  peer- 
reviewed  journals.  This  search  yielded  552  potentially  rele¬ 
vant  articles  for  evaluation.  In  addition,  we  performed  a  hand 
search  of  literature  in  relevant  journals,  consulted  with  ex¬ 
perts  about  relevant  TMH  publications,  and  reviewed  refer¬ 
ences  cited  in  articles,  review  articles,  and  meta-analyses 
obtained  in  the  initial  search;  these  procedures  yielded  20 
additional  sources  that  were  included  in  the  full  review. 

INCLUSION  CRITERIA 

We  established  the  following  inclusion  criteria  for  review¬ 
ing  the  identified  subset  of  studies,  all  of  which  had  to  be  met 
in  order  to  be  included  in  our  full  review:  (1)  study  design 
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included  both  TMH  and  in-person  treatment  groups;  (2)  TMH 
intervention  composed  of  direct,  synchronous  communica¬ 
tion  with  a  mental  health  provider;  (3)  TMH  treatment  was 
primarily  concerned  with  a  psychological  treatment  involving 
a  psychotherapeutic  relationship  (not  exclusively  psychiatry) ; 
and  (4)  outcome  variables  included  measure  of  treatment 
satisfaction  and/or  therapeutic  alliance. 

FULL  REVIEW  AND  ARTICLE  ABSTRACTION 
The  first  author  (M.A.J.-G.)  reviewed  the  titles  and  abstracts 
of  the  552  articles  for  relevance  and  potential  fit  with  our 
inclusion  criteria  [Fig.  1).  This  process  yielded  101  articles 
eligible  for  full  text  review  to  determine  if  these  studies  ex¬ 
plicitly  met  our  inclusion  criteria;  we  also  reviewed  fully  and 
applied  the  inclusion  criteria  to  the  20  articles  obtained  from 
hand  searching  and  related  methods  performed  during  the 
review  process.  The  first  two  authors  (M.A.J.-G.  and  L.D.P.) 
independently  evaluated  the  full  text  of  these  articles  using  a 
standardized  criteria  form  and  discussed  any  inconsistencies 
regarding  inclusion  decisions  to  form  consensus.  This  process 
identified  18  studies  focusing  on  psychotherapeutic  TMH 
treatment;  however,  upon  further  inspection  and  contact  with 
the  study  authors,  four  studies  were  re-analyses  of  data  pre¬ 
viously  used  in  other  studies  already  included  in  our  review 
and  so  were  excluded.  Thus,  14  articles  were  included  in 
our  final  review.  We  abstracted  relevant  information  from 
these  studies  using  a  coding  form,  including  sample  size, 
telepsychology  modality,  diagnoses  treated,  client  population, 
type  of  intervention,  satisfaction  and  therapeutic  alliance 
measures,  and  any  other  factors  related  to  patient  perceptions 
of  treatment.  These  articles  are  summarized  in  Table  1 . 


Fig.  l.  Flow  diagram  detailing  study  selection  for  review. 


Results 

STUDY  DESCRIPTIONS 

Nine  of  the  studies  used  randomized  controlled  trial  de¬ 
signs,  of  which  four  were  noninferiority  trials.  The  remaining 
five  studies  relied  on  nonexperimental  designs  such  as  active 
control  group  designs  without  randomization,  posttreatment 
quit-rate  comparisons,27  or  multiple  baseline  through  random 
assignment.28  Study  samples  were  drawn  from  both  rural  and 
urban  populations  from  the  United  States  (n  =  9),  Canada 
[n  =  4),  the  United  Kingdom  [n—  1),  and  Australia  [n=  1);  all 
studies  targeted  adults.  Most  of  the  studies  involved  treat¬ 
ments  delivered  via  VTC  (>?  =  13),  whereas  two  studies  were 
telephone-based  TMH.  Although  most  of  the  sample  sizes 
could  be  considered  small  (for  unique  samples:  median  =46; 
mean  =  84.2),  the  nonexperimental  group  VTC-based  smoking 
cessation  study  of  Carlson  et  al.27  included  554  people.  The 
most  common  therapeutic  orientation  evaluated  was  CBT 
(11  studies). 

PATIENT  PERCEPTIONS  OF  SATISFACTION 
AND  ALLIANCE 

Four  studies  used  custom  scales  developed  for  individ¬ 
ual  investigations,  whereas  others  used  standardized  mea¬ 
sures  such  as  the  Working  Alliance  Inventory,29  the  Session 
Evaluation  Questionnaire,30  or  the  Charleston  Psychiatric 
Outpatient  Satisfaction  Scale-VA.18  Findings  indicated  no 
statistically  significant  condition  effects  for  those  studies  that 
directly  compared  patient  ratings  of  satisfaction  with  psy¬ 
chotherapeutic  treatment  between  in-person  and  TMH  con- 
ditions.4'7'31-35  Similarly,  six  analyses  found  no  significant 
differences  between  modalities  on  patient  ratings  of  thera¬ 
peutic  alliance.3 1,34,36-39  Four  other  studies 
reported  no  significant  differences  for  related 
process  measures  or  custom  measures  captur¬ 
ing  some  element  of  the  treatment  experience 
related  to  therapeutic  relationship  or  client 
perceptions  of  treatment.4’7'37,39 

Four  studies  reported  a  statistically  signifi¬ 
cant  difference  between  conditions  on  more 
relationally  oriented  measures  such  as  alliance 
between  client  and  therapist.  The  in-person 
condition  patients  of  Morland  et  al.7  reported 
higher  ratings  for  group  therapeutic  alliance 
during  anger  management  treatment  compared 
with  patients  in  the  TMH  condition.  Although 
the  study  of  Greene  et  al.40  is  a  later  analysis  of 
the  randomized  controlled  trial  of  Morland 
et  al.,7  both  will  be  discussed  here  given  the 
distinct  finding  of  Greene  et  al.40  based  on 
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Table  1.  Characteristics  of  Included  Studies 
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Group  Therapy  Alliance  Scale  41  subscales.  Greene  et  al.40 
reported  lower  Group  Therapy  Alliance  Scale  Self-Leader  al¬ 
liance  subscale  scores  in  the  VTC  condition,  suggesting  lower 
perceived  connection  between  individual  patients  and  the 
group  therapist.  However,  they  also  noted  that  patients  in  the 
VTC  group  reported  relatively  high  levels  on  the  Likert  scale 
used  (mean  was  4.2  out  of  a  possible  5  points).  Similarly, 
participants  in  the  in-person  condition  of  Frueh  et  al.4  re¬ 
ported  greater  comfort  in  talking  with  the  group  leader.  Lastly, 
Ertelt  et  al.37  reported  that  providers  not  involved  in  treatment 
who  rated  sessions  of  individual  CBT  treatment  for  bulimia 
nervosa  indicated  higher  Working  Alliance  Inventory  ratings 
for  the  in-person  group  compared  with  VTC  delivery.  How¬ 
ever,  it  is  important  to  note  that  the  actual  patient  ratings  were 
comparable  between  conditions.  Although  insufficient  data 
were  available  to  make  firm  conclusions,  the  only  significant 
differences  in  the  studies  reviewed  involved  higher  patient 
ratings  of  alliance  and  comfort  in  group  treatment  models  for 
in-person  treatment. 

THERAPEUTIC  ORIENTATION  AND  TREATMENT  FORMAT 

Across  CBT  studies,  there  was  strong  support  for  compa¬ 
rable  levels  of  patient  satisfaction  between  conditions,  par¬ 
ticularly  for  individual  psychotherapy.  Results  also  suggested 
similar  satisfaction  and  patient-provider  alliance  scores 
across  the  limited  sample  of  non-CBT  interventions, 28,32,34,35 
although  additional  research  is  needed  for  other  therapeutic 
orientations  (e.g.,  interpersonal  psychotherapy,  motivational 
interviewing).  Three  of  the  four  studies  included  in  this  review 
that  found  differences  between  TMH  and  in-person  conditions 
were  evaluations  of  group  interventions.  This  suggests  that 
psychotherapy  format  may  be  a  significant  factor  in  patient 
perceptions  of  the  treatment  experience. 

DIAGNOSTIC  CATEGORY 

Across  these  categories,  the  results  consistently  supported 
the  conclusion  that  both  in-person  and  remote  care  demon¬ 
strated  high  overall  levels  of  patient  satisfaction.  Although 
some  interesting  deviations  from  this  pattern  exist,  the  evi¬ 
dence  suggests  that  the  presenting  problem  does  not  strongly 
influence  satisfaction  ratings  for  the  method  of  treatment 
delivery.  The  most  commonly  investigated  diagnostic  categories 
included  depression  (w  =  3)  and  posttraumatic  stress  disor¬ 
der  (w  =  6),  and  studies  focused  on  these  two  disorders  found 
comparable  ratings  of  treatment  satisfaction  between  remote 
and  in-person  modalities. 

Frueh  et  al.4  reported  that  individuals  receiving  in-person 
care  showed  higher  levels  of  comfort  for  talking  with  their 
therapist  compared  with  those  in  the  remote  care  condition  at 


their  postassessment.  However,  this  difference  may  be  more 
related  to  the  group  treatment  format. 

In  two  studies  focused  on  substance-related  issues,27,32  the 
authors  noted  that  patient  dissatisfaction  arose  when  experi¬ 
encing  technical  difficulties  with  the  TMH  equipment.  It  is  in¬ 
teresting  that  King  et  al.32  reported  that  patients  receiving 
group  relapse  prevention  treatment  over  TMH  reported  an  in¬ 
creased  perception  of  privacy  compared  with  those  participat¬ 
ing  in  in-person  treatment.  This  may  have  been  because  TMH 
participants  could  not  see  images  of  one  another  and  only  audio 
was  shared  between  participants.  In  the  study  on  group  treat¬ 
ment  for  anger  management,7  the  in-person  group  produced 
higher  ratings  of  therapeutic  alliance  than  the  VTC  group. 

Lastly,  although  patients  in  the  study  of  Mitchell  et  al. 12  had 
comparable  alliance  scores  between  remote  and  in-person 
conditions,  their  treatment  providers  rated  their  own  levels 
higher  for  in-person  treatment  delivery.  These  authors  suggest 
that  this  finding  may  have  resulted  from  providers  being  more 
focused  on  how  to  apply  the  treatment  across  a  novel  mo¬ 
dality,  whereas  patients  were  more  focused  on  the  content  of 
the  intervention  itself,  regardless  of  modality. 

Discussion 

The  results  of  this  systematic  review  show  that,  in  general, 
patient  ratings  of  satisfaction  with  psychotherapeutic  inter¬ 
ventions  and  therapeutic  alliance  in  treatment  are  comparable 
between  remote  TMH  and  in-person  delivery.  However,  it  is 
also  important  to  note  that  the  research  conducted  to  date  has 
generally  involved  small  sample  sizes,  which  may  limit  con¬ 
clusions.  Other  treatment  factors  such  as  rapport  and  pro¬ 
vider  expertise,  as  well  as  patient  factors  such  as  personality 
variables  and  treatment  preferences,  are  also  likely  to  be 
important  variables  associated  with  patient  satisfaction  and 
therapeutic  alliance.  In  addition,  the  efficacy  of  the  specific 
treatment  protocols  used  to  guide  treatment  may  be  connected 
to  patient  perceptions  of  treatment  and,  ultimately,  their  sat¬ 
isfaction  with  the  treatment  process  and  clinical  outcome,  re¬ 
gardless  of  the  means  through  which  treatment  is  delivered. 
Furthermore,  in  the  studies  reviewed  that  showed  different 
satisfaction  levels  between  modalities,  these  findings  appear 
to  be  associated  with  more  modifiable  factors  such  as  the  use 
of  group  treatment  or  technological  factors  that  influence  VTC 
quality.  These  results  imply  that  additional  preparation  and 
focus  on  patient  experience  in  treatment  may  be  useful  in 
group  treatments  delivered  via  TMH.  Additionally,  providers 
should  follow  established  guidelines,  such  as  those  published 
by  the  American  Telemedicine  Association, 1  in  selecting  and 
setting  up  the  technology  to  be  used  in  synchronous  psy¬ 
chotherapeutic  TMH  treatment. 
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Care  providers  must  work  with  patients  to  weigh  the  pros  and 
cons  of  using  TMH  services  and  determine  what  alternatives 
may  be  appropriate  (e.g.,  traveling  to  a  distant  clinic  for  ser¬ 
vices).  In  some  scenarios,  it  may  be  that  treatment  through  TMH 
allows  for  the  greatest  access  to  care  while  also  risking  reduced 
satisfaction  and  engagement  with  treatment  due  to  external 
factors  such  as  Internet  bandwidth.  However,  this  risk  may  be 
warranted  when  the  alternative  to  TMH  is  to  travel  great  dis¬ 
tances  (e.g.,  to  a  metropolitan  center),  increasing  the  time  and 
cost  of  seeking  and  receiving  treatment— especially  among 
populations  who  live  on  a  fixed  income,  whose  healthcare  re¬ 
quires  travel  to  a  specific  center  (e.g.,  Veterans  Administration 
Medical  Center),  or  who  cannot  afford  the  luxury  of  taking 
time  off  work,  arranging  childcare,  and  commuting  to  and  from 
an  appointment  on  a  recurring  basis.  Previous  research  has 
highlighted  the  importance  of  patient  preference  and  choice  of 
treatment,  as  these  factors  have  relationships  with  both  patient 
satisfaction  and  clinical  outcomes.42  Because  of  these  consid¬ 
erations,  future  research  must  begin  to  consider  the  patient’s 
choice  as  a  contributing  factor  in  both  the  efficacy  and  satis¬ 
faction  associated  with  treatment  delivered  via  TMH  and 
whether  responses  to  treatment  are  modulated  by  one’s  match 
between  the  desired  mode  of  treatment  delivery  and  the  re¬ 
ceived  mode  of  treatment  delivery. 

Further  research  is  needed  to  more  fully  assess  the  role  of 
intervention  format  (e.g.,  group  therapy,  individual  therapy)  in 
patient  satisfaction  and  therapeutic  alliance  given  the  small 
number  of  studies  available  from  which  to  draw  inferences. 
Additionally,  investigations  seeking  to  understand  how  indi¬ 
viduals  engage  with  telehealth  treatment  are  needed,  as  well  as 
research  on  perceptions  of  treatment  experience  for  people  with 
other  specific  psychiatric  diagnoses.  Richardson  et  al.20  em¬ 
phasized  shortcomings  in  the  research  literature  on  patient 
satisfaction  with  TMH  services  by  highlighting  the  need  for 
evaluating  satisfaction  with  the  TMH  modality  as  a  distinct 
construct  from  general  satisfaction  with  treatment.  Researchers 
should  define  patient  satisfaction  with  treatment  carefully,  as 
some  have  questioned  whether  findings  suggesting  comparable 
satisfaction  ratings  are  primarily  based  on  the  treatment  pro¬ 
gram  instead  of  being  tied  to  the  modality  being  used  (i.e.,  VTC). 
As  our  review  highlights,  researchers  have  been  inconsistent  in 
the  types  of  satisfaction  measures  used  across  studies,  making  it 
more  difficult  to  objectively  compare  studies  and  evaluate 
patient  satisfaction  as  an  outcome.  Use  of  these  standard  sat¬ 
isfaction  measures  can  be  useful  not  only  for  research,  but 
should  be  part  of  all  TMH  services  programs  to  help  inform 
program  evaluation  and  improvement.  The  assessment  of 
provider  satisfaction  with  TMH  may  also  be  useful  for  making 
improvements  to  TMH  programs. 


PRACTICE  RECOMMENDATIONS 

Several  studies  examined  for  this  review  suggest  that 
training  for  clinicians  on  TMH  delivery  of  psychotherapeutic 
services  could  be  implemented  to  increase  clinician  comfort 
and  confidence  with  this  modality,  which  in  turn  may  affect 
patient  satisfaction.34,37,43  It  is  also  important  to  explain  the 
TMH  process  and  expectations  with  patients  and  to  encourage 
questions  and  discussion  of  concerns  before,  during,  and  after 
TMH  sessions.  Similarly,  it  can  be  useful  to  emphasize  the 
practical  aspects  of  TMH  delivery,  such  as  technical  support 
with  computer  equipment  and  specific  instruction  on  using 
the  computer  and  software  involved32  as  well  as  remote  room 
setup  and  adequate  lighting.12  Clinicians  may  also  need  to 
adapt  their  delivery  to  overcome  potential  barriers  in  remote 
modalities,  for  example,  by  repeating  discussion  of  psychoe- 
ducational  content  and  directing  attention  toward  related 
materials  (e.g.,  handouts)  in  treatment.35 

Appropriate  involvement  of  an  assistant  or  mental  health 
professional  available  at  the  remote  treatment  site  may  also 
affect  patient  satisfaction  and  can  be  an  important  resource 
for  clinicians  in  managing  suicide  risk.12  Local  assistants 
can  be  used  to  increase  engagement  in  groups  and  facilitate 
practical  aspects  of  treatment  such  as  transmitting  forms  or 
assessments  to  the  clinician.27,39  Facilitators  may  also  be  in¬ 
strumental  ensuring  that  technical  problems  are  minimized  by 
testing  equipment  and  connections  as  well  as  assisting  with 
troubleshooting  procedures. 

Conclusions 

The  quality  of  patient  satisfaction  and  the  therapeutic  al¬ 
liance  are  essential  to  both  the  effectiveness  of  TMH-based 
treatments  and  to  the  acceptance  of  TMH  services  among  both 
patients  and  care  providers.  Consistent  use  of  standardized 
measures  of  patient  treatment  satisfaction  in  research  studies 
and  in  general  clinical  practice  is  a  necessity  to  accurately 
assess  these  important  variables.  TMH-based  services  con¬ 
tinue  to  show  great  promise  for  addressing  barriers  to  care, 
including  geographic  limitations  and  shortages  of  healthcare 
providers.  The  application  of  best  practices  that  help  to  im¬ 
prove  patient  satisfaction  and  to  optimize  the  quality  of  other 
therapeutic  process  variables  with  TMH  will  serve  to  expand 
the  wider  adoption  of  TMH  services  and  assure  the  best  pos¬ 
sible  quality  of  care. 
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ABSTRACT 

The  effective  assessment  and  management  of  suicidal  patients  is  an  essential 
component  of  telehealth-based  care.  With  this  article,  we  describe  how  we 
have  implemented  procedures  for  the  ongoing  assessment  and  management 
of  suicide  risk  in  a  clinical  trial  that  compares  in-office  treatment  to  home- 
based  treatment  delivered  via  web-cam  to  U.S.  military  service  members 
and  veterans  with  depression.  We  describe  our  safety  protocol  and  how  it 
was  adapted  from  current  recommended  best  practices,  published  guide¬ 
lines,  and  local  requirements  for  managing  patient  safety  during  home-based 
telepractice.  We  conclude  with  discussion  of  other  key  safety  issues  asso¬ 
ciated  with  telepractice.  The  topics  discussed  are  relevant  to  all  mental 
health  practitioners  who  are  interested  in  clinical  telepractice  services. 

[Int'l.  J.  Psychiatry  in  Medicine  2014;48:19-31) 
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INTRODUCTION 

The  effective  assessment  and  management  of  suicidal  patients  is  a  critical  com¬ 
ponent  of  both  conventional  in-office  care  and  telehealth-based  care.  There 
are  aspects  of  telepractice,  however,  that  require  additional  considerations  for 
managing  the  safety  of  patients  who  are  or  become  high  risk  for  suicidal  behavior 
while  under  care.  In  clinically  supervised  settings  (e.g.,  a  provider’s  office, 
clinics,  hospitals,  etc.),  clinical  staff  are  typically  available  to  assist  during  a 
clinical  crisis  by  coordinating  emergency  services,  providing  consultation,  or 
escorting  a  patient  to  the  emergency  department.  The  same  is  not  necessarily  true 
when  care  is  delivered  to  settings  that  do  not  have  clinical  supervision  at  the 
remote  site  (e.g.,  a  patient’s  home).  Thus,  management  of  suicide  risk  and  patient 
safety  during  telepractice  involves  additional  considerations  and  requirements. 
A  particular  concern  is  what  to  do  in  situations  where  a  patient  expresses  intent 
to  harm  him/herself  at  the  end  of  a  telehealth  session  or  before  unexpectedly 
disconnecting  from  the  session  [1],  In  order  to  effectively  manage  this  type  of 
crisis  or  other  emergency  situations  (e.g.,  medical  emergencies),  it  is  necessary 
for  telehealth  providers  to  have  a  pre-planned  process  in  place. 

The  assessment  and  management  of  patients’  suicidal  behavior  while  under  care 
can  be  a  very  difficult  and  stressful  experience  for  mental  health  clinicians  [2,  3]. 
In  the  case  of  clinical  telepractice,  stress  and  anxiety  can  be  exacerbated  by  the 
fear  of  having  less  control  of  the  situation,  unfamiliarity  with  safety  procedures, 
technology  issues,  and  concerns  about  liability  [1].  The  issue  of  potential  liability 
is  of  particular  concern  for  many  mental  health  practitioners  because  inadequate 
suicide  risk  management  can  result  in  licensure  complaints  and/or  malpractice 
lawsuits.  Due  to  the  ethical  and  legal  responsibilities  mental  health  practitioners 
have  toward  patients,  liability  can  occur  from  even  the  briefest  of  patient  contacts. 
The  use  of  technology  to  deliver  care  (e.g.,  video  conferencing,  e-mail,  web  chat) 
introduces  additional  ways  that  a  professional  relationship  can  fonn  and  with  it 
raises  responsibilities  to  assess  and  manage  suicide  risk.  The  anxiety  and  concern 
about  liability  issues  among  individual  practitioners  and  healthcare  organizations 
as  a  result  of  these  additional  methods  of  delivering  care  can  present  a  barrier  to 
the  wider  adoption  of  telehealth-based  services. 

Several  organizations,  such  as  the  American  Psychological  Association  (APA) 
and  the  American  Telemedicine  Association  (AT A)  have  issued  guidelines  that 
include  provisions  for  patient  safety  management  during  telepractice  [4,  5].  The 
American  Psychiatric  Association  does  not  have  its  own  telepractice  guidebook, 
although  the  association  refers  its  constituents  interested  in  telemedicine  to 
the  ATA  guidelines  [6],  Clinical  best  practices  regarding  safety  management 
specific  to  telemental  health  have  also  been  published  [7,  8],  as  have  telemental 
health  guidebooks  that  address  patient  safety  [9].  Although  the  available  guide¬ 
lines,  extant  telepractice  literature,  and  the  general  suicide  risk  management 
literature  [2,  10-14]  provide  frameworks  for  effective  risk  management,  the 
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literature  is  limited  in  detailed  information  regarding  real-world  implementa¬ 
tion  of  suicide  risk  assessment  and  management  protocols  for  telehealth-based 
services,  particularly  to  clinically  unsupervised  settings  such  as  the  home. 

With  this  article  we  address  this  limitation  by  describing  how  we  have  trans¬ 
lated  safety  guidelines,  practice  recommendations,  and  local  requirements  into 
ongoing  assessment  and  management  of  suicidal  risk  as  part  of  a  clinical  trial  that 
compares  in-office  treatment  to  care  delivered  to  the  home  via  web-cam  to  patients 
with  depression.  We  describe  our  safety  protocol,  suicide  risk  assessment  and 
management  procedures,  and  how  we  have  applied  our  safety  protocol  to  mitigate 
risk  during  telepractice.  We  recognize  that  our  protocol  is  limited  by  the  specific 
clinical  setting  and  population  (clinical  research  at  U.S.  military  and  VA 
Hospitals);  however,  the  principles  and  issues  that  we  describe  have  applic¬ 
ability  to  other  clinical  telepractice  settings.  We  do  not  elaborate  on  the  issues 
surrounding  licensure  and  liability  as  these  have  been  adequately  covered 
elsewhere  [15]. 


CLINICAL  TRIAL  DESCRIPTION  AND 
SAFETY  PROCEDURES 

The  clinical  trial  (ClinicalTrials.gov  Identifier  #NCT01599585)  is  being  con¬ 
ducted  at  the  U.S.  Department  of  Defense’s  National  Center  for  Telehealth  and 
Technology  (T2)  located  at  Joint  Base  Lewis-McChord  (JBLM)  in  Washington 
State  and  at  the  Portland  VA  Healthcare  System  in  Oregon.  The  aim  of  the  trial 
is  to  compare  in-office  to  home-based  delivery  of  an  abbreviated  (eight-session) 
version  of  the  revised  Brief  Behavioral  Activation  for  Depression  protocol 
(BATD-R)  [16].  BATD-R  is  a  behavioral  reinforcement-based  treatment  that 
has  received  extensive  empirical  support  as  a  treatment  for  depression  [17]. 
Patients  in  our  trial  include  both  U.S.  military  personnel  and  veterans  who  either 
self-refer  or  are  referred  to  the  study  by  behavioral  health  providers  at  each 
respective  site.  While  home-based  telemental  health  treatments  are  already  being 
expanded  in  the  VA  Health  System,  home-based  telemental  healthcare  is  not 
presently  the  standard  of  care  in  the  U.S.  military.  Thus,  the  primary  purpose  of 
the  trial  is  to  examine  the  feasibility,  safety,  and  effectiveness  of  home-based 
telemental  health  in  the  military  setting  to  inform  policy  for  broader  imple¬ 
mentation  of  home-based  treatments.  The  study  protocol  adheres  to  the  principles 
and  recommendations  of  the  World  Medical  Association,  Declaration  of  Helsinki 
Ethical  Principles  for  Medical  Research  Involving  Human  Subjects,  as  well  as 
all  applicable  Codes  of  Federal  Regulation  and  Department  of  Army  Regulations. 
This  study  was  approved  by  the  Madigan  Army  Medical  Center  Institutional 
Review  Board  and  the  Army  Human  Research  Protection  Office. 

For  our  trial,  eligible  participants  are  randomized  to  either  the  in-office  or 
in-home  treatment  groups.  All  participants  are  provided  with  eight  sessions  of 
BATD  that  follows  a  treatment  manual  [17],  Participants  in  both  intervention 
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groups  follow  the  same  assessment  schedule  with  clinical  assessments  at  baseline 
(before  first  treatment  session),  mid-treatment  (week  4),  1-week  post-treatment, 
and  3-months  post-treatment.  A  detailed  description  of  the  trial’s  methodology 
is  published  elsewhere  [16]. 

Safety  Protocol 

The  essential  components  of  our  safety  protocol  planning  steps  and  processes 
are  shown  in  Table  1.  The  safety  protocol  was  designed  in  accordance  with  the 
professional  guidelines  and  best  practices  literature  available  at  the  time  [5,  7]  and 
is  consistent  with  the  most  recent  applicable  guidelines  from  both  the  American 
Telemedicine  Association  [18]  and  American  Psychological  Association  [4], 
Our  safety  protocol  was  made  into  a  formal  written  plan  that  is  part  of  our 
trial’s  research  protocol. 

The  development  of  our  safety  protocol  began  with  review  of  applicable  local 
regulatory  requirements  and  guidance.  Our  study  clinicians  (clinical  psychol¬ 
ogists)  are  credentialed  providers  at  Federal  facilities;  Madigan  Army  Medical 
Center  (MAMC)  and  Portland  VA  Medical  Center  (PVAMC).  Thus,  the  standard 
operating  procedures  (SOP)  of  the  Army  and  PVAMC  were  reviewed  and 
included  in  our  plan.  This  review  included  examination  of  duty-to-wam  and 
mandated  safety  reporting  requirements.  For  active-duty  military  personnel,  their 
command  must  be  notified  when  the  service  member’s  safety  is  a  concern. 
Thus,  these  patients  are  asked  to  provide  command  contact  information.  The  use 
of  support  persons  is  a  recommended  approach  to  telehealth  safety  planning 
[5,  7].  Depending  on  the  type  of  clinical  setting,  an  additional  support  person  to 
assist  with  coordination  in  emergencies  could  include  another  treatment  pro¬ 
vider,  other  designated  staff  at  the  patient  site,  a  family  member,  or  a  local 
community  contact  who  knows  the  patient  and  agrees  to  remain  reachable  during 
telehealth  sessions  [19].  Thus,  we  ask  patients  at  both  sites  to  identify  another 
person  (e.g.,  family  member,  partner,  or  friend)  who  can  be  notified  in  case  of 
an  emergency.  Patients  at  both  sites  are  asked  to  complete  a  site  specific  release 
of  information  form  so  that  the  third  party  can  assist  in  cases  of  emergency 
or  imminent  risk.  These  processes  are  discussed  with  our  patients  during  the 
informed  consent  process  upon  entry  into  the  clinical  trial. 

Screening  and  Assessment 

Telepractice  guidelines  uniformly  urge  clinicians  to  determine  appropriateness 
of  each  patient  for  telehealth  care  prior  to  initiating  services  [4,  5].  “Appropriate¬ 
ness”  varies  across  contexts  based  on  several  factors  including  technology  con¬ 
siderations,  patient  needs  and  preferences,  and  administrative  regulations  [1,  7].  In 
mental  health  care,  suicide  risk  assessment  is  a  critical  aspect  of  determining 
appropriateness  for  varied  treatment  modalities.  In  our  trial,  suicide  risk  assess¬ 
ment  begins  during  the  initial  in-person  screening  of  patients.  We  first  administer 
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Table  1 .  Overarching  Safety  Plan  Steps  and  Process 


Safety  planning  step 

Process 

Practice  within  institu¬ 
tional  rules,  regula¬ 
tions,  and  state  laws 

•  Review  local  and  health  systems  regulations. 

•  Providers  receive  training  and  supervision  on  pertinent  institu¬ 
tional  and  legal  regulations  for  providing  telehealth-based  care. 

Determine  appro¬ 
priateness  of 
telehealth-based  care 

•  Conduct  pre-treatment  clinical  assessment  and  suicide  screen  to 
determine  risks,  contraindications,  etc. 

Ensure  adequacy  of 
home-environment, 
technology,  and 
devices 

•Test  infrastructure  for  adequate  bandwidth. 

•  Assess  quality  of  environment  (e.g.,  sound,  lighting,  privacy,  etc.) 
and  equipment  (e.g.,  computer,  microphones,  cameras,  etc.). 

•  Ensure  tech  support  plan  and  materials  (troubleshooting 
guides). 

•  Plan  for  maintaining  privacy  at  patient’s  end. 

Conduct  site 

assessments  and 
establish  procedures 

•  Obtain  alternative  contact  numbers  from  patient. 

•  Obtain  patient’s  local  emergency  contact  information  and  confirm 
with  EMS  agency  (using  non-emergency  line). 

•  Identify  local  collaborators  (e.g.,  Patient  Support  Person)  that  can 
be  called  to  support  patient  safety  during  crisis. 

•  Obtain  needed  authorizations  to  release  information. 

•  Provider  ensures  that  he/she  has  access  to  a  secondary  phone 
line  in  the  clinical  room  during  appointments. 

•  Have  secondary  staff  available  during  appointments  to 
coordinate  with  EMS,  if  needed. 

Discuss  roles  and 
responsibilities 
with  patient 

•  Discuss  technical  troubleshooting  with  patient  and  have  an 
agreed  upon  method  for  re-establishing  contact  during  service 
disruption  (e.g.,  via  telephone). 

Evaluate  patient  risk 
during  and  after 
treatment 

•  Monitor  psychiatric  symptom  levels. 

•Assess  for  presence  and/or  change  in  suicidality. 

•  Monitor  relevant  changes  in  patient’s  home  environment. 

•  (If  indicated  by  risk  level)  Develop  multi-step  safety  plan  and 
provide  patient  with  a  copy  of  the  plan.  Lead  a  direct  and  candid 
discussion  about  patient’s  access  to  firearms  or  other  lethal 
means,  and  generate  strategies  to  restrict  access.  Determine 
how  transportation,  if  necessary,  will  be  handled  and  whether  to 
utilize  a  local  collaborator. 

•  (If  indicated  by  risk  level)  T ry  to  remain  connected  to  patient  via 
VTC  while  coordinating  involvement  of  EMS  by  telephone. 

Involve  secondary  staff  and  notify  third  parties  as  warranted. 

Note:  This  list  is  based  on  that  presented  by  Luxton  and  colleagues  [7], 
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the  Structured  Clinical  Interview  for  the  DSM-IV  Axis  I  Disorders,  Research 
Version,  Patient  Edition  (SCID-I/P)  [20]  to  determine  initial  study  eligibility  and 
to  screen  for  current  suicidal  ideation  and  past  self-injurious  events.  Potential 
patients  are  ineligible  for  the  trial  if  they  report  a  suicide  attempt  during  the 
past  6  months  or  if  they  have  current  ideation  with  stated  intent.  These  exclusion 
criteria  may  eliminate  the  highest  risk  patients  that  are  encountered  in  other 
clinical  settings;  however,  these  exclusion  criteria  were  deemed  necessary  for  this 
study  because  home-based  care  is  not  the  standard  of  care  in  our  setting  (and 
thus  deemed  experimental  by  our  IRB). 

As  part  of  our  overall  safety  plan,  we  use  the  Standard  Operating  Procedure 
(SOP)  for  the  Assessment  and  Management  of  Suicide  and  Homicide  Risk  in 
Active  Duty  Service  Members  that  is  used  at  MAMC  [21].  This  official  SOP 
was  updated  during  the  course  of  our  study,  therefore  we  updated  our  procedures 
to  remain  consistent  with  the  SOP.  The  SOP  is  based  on  information  from 
several  sources  including  the  VA/DoD  Clinical  Practice  Guidelines  for  Assess¬ 
ment  and  Management  of  Patients  at  Risk  for  Suicide  [22],  U.S.  Air  Force 
Guidelines  for  managing  suicide  behavior  [23],  and  several  other  DoD  policies 
and  procedures.  The  same  SOP  guides  assessments  completed  in-person  or 
during  telehealth  sessions. 

The  SOP  specifies  a  five-step  process.  Step  one  consists  of  a  screen  for  the 
presence  of  suicidal,  violent,  or  homicidal  ideation,  intent,  or  behavior.  If  the 
screen  suggests  presence  of  any  risk,  a  full  assessment  interview  is  administered 
(step  two)  that  assesses  for  frequency,  intensity,  and  duration  of  ideation,  content 
of  thoughts  and/or  plans,  impulsivity,  history  of  suicidal  and  violent  behavior, 
and  other  warning  signs,  risk,  and  protective  factors.  At  the  third  step,  clinicians 
integrate  all  information  gathered  from  the  assessment  interview  and  compare 
that  information  to  the  SOP’s  guidelines  in  order  to  arrive  at  and  document 
the  current  level  of  risk  (i.e.,  not  at  elevated  risk,  low  risk,  intermediate  risk,  or 
high  risk).  The  general  descriptions  of  the  levels  of  risk  and  associated  clinical 
interventions  specified  in  the  SOP  are  shown  in  Table  2.  In  step  four,  clinicians 
are  to  document  and  provide  rationale  for  the  clinical  responses  provided.  Finally, 
in  step  five  of  the  risk  assessment,  clinicians  develop  and  document  safety 
plans  with  all  patients  with  any  elevation  of  risk.  Safety  plans  can  vary  based  on 
idiographic  factors;  however,  the  SOP  encourages  use  of  a  safety  plan  to  assist 
patients  in  identification  of  healthy  coping  strategies  to  be  used  when  distressed, 
people  to  contact  for  additional  support,  ways  to  reduce  risk  in  their  environments 
(i.e.,  limiting  substance  use  and  restricting  access  to  means),  emergency/crisis 
response  contact  numbers,  and  making  a  commitment  to  living  and  to  engage 
in  treatment. 

In  our  trial,  the  suicide  risk  assessment  SOP  is  administered  both  at  the 
intake/screening  assessment  and  again  during  the  first  treatment  session.  It  is  also 
administered  during  subsequent  assessment  and/or  treatment  sessions  as  needed. 
That  is,  if  a  patient  were  to  indicate  a  change  in  the  severity  or  frequency  of 
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Table  2.  Determination  of  Suicide  and  Homicide  Risk  Level 


Risk  level 

Criteria  used  to  determine  risk3 

Clinical  response5 

Not  at 

elevated  risk 

Denial  of  recent  violent  or 
suicidal  ideation,  intent,  plans, 
or  preparations. 

No  history  of  violent  behavior 
or  suicide  attempt  in  the 
previous  2  years. 

No  change  necessary  in  routine 
outpatient  clinical  practice. 

Provide  contact  information  for  emer¬ 
gency  responders. 

May  consider  devising  safety  plan  for 
highly  distressed  patients. 

Low  risk 

Endorsement  of  recent  violent, 
homicidal,  or  suicidal  ideation 
without  intent  to  act  or  devise. 

Frequency  and  duration  of 
ideation  is  low. 

No  evidence  of  preparations  or 
difficulty  controlling  impulses. 

No  violent  behavior  or  suicide 
attempt  in  the  previous  year. 

Establish  a  safety  plan  with  patient 
that  addresses  coping  strategies, 
contact  information  for  social 
supports,  means  restriction  and 
limiting  of  substance  use,  and 
emergency  contact  information. 

Elicit  a  commitment  to  living  and  to 
engaging  in  treatment. 

Intermediate 

risk 

Endorsement  of  current  homi¬ 
cidal  or  suicidal  ideation 
without  intent  to  act  or  difficulty 
controlling  impulses. 

Frequency  and  duration  of 
ideation  is  moderate  to  high. 

No  recent  violent  behavior, 
suicide  attempt,  or  preparations. 

Take  precautions  of  low  risk  and 
consider  increasing  frequency 
and/or  intensity  of  contact  to 
>  one  time  per  week. 

Engage  in  peer  consultation  to  share 
and  track  decision-making  process 
and  determine  need  for  internal  and 
external  reporting/disclosures  and 
means  restrictions. 

High  risk 

Endorsement  of  persistent 
homicidal  or  suicidal  idea¬ 
tion  with  a  plan  or  intent  to  act 
on  a  plan,  and  difficulty  con¬ 
trolling  impulses. 

Or,  recent  violent  act,  suicide 
attempt,  or  preparations. 

Engage  lower  level  precautions 
(including  development  of  detailed 
safety  plan  with  means  restriction) 
and  increase  treatment  intensity. 
Strongly  consider  implementing 
emergency  response  to  arrange  for 
safe  transport  of  patient  for  evalua¬ 
tion  and  possible  inpatient  hos¬ 
pitalization. 

Initiate  reporting/disclosure  processes 
as  indicated. 

Note:  The  contents  of  this  table  are  based  on  the  MAMC  risk  assessment  SOP  [20]. 
aln  addition  to  these  criteria,  clinical  judgment  is  used  to  integrate  additional  known  risk 
factors  (e.g.,  agitation,  significant  psychosocial  stressors,  hopelessness)  and  protective 
factors  (e.g.,  interpersonal  connections,  help  seeking,  optimism)  to  ultimately  arrive  at  and 
document  a  current  risk-level  determination.  Further,  if  a  patient  has  greater  than  one 
previous  suicide  attempt  or  violent  act,  risk  level  automatically  advances  at  least  one 
level.  5These  clinical  responses  reflect  the  minimum  indicated  response.  Clinical  judgment  is 
always  part  of  risk  evaluation  and  clinicians  may  elect  to  engage  a  higher  level  of  inter¬ 
vention  if  deemed  appropriate. 
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suicidal  ideation,  the  treatment  provider  would  again  assess  for  suicide  risk  per 
the  SOP.  If  a  patient  is  assessed  to  be  at  not  elevated  or  low  risk,  s/he  would 
not  be  assessed  again  until  the  final  treatment  session.  Patients  at  or  above 
intermediate  risk  are  assessed  during  each  treatment  session  until  level  of  risk 
decreases  below  the  intermediate  risk  threshold. 

Patient  Monitoring  and  Telehealth  Session  Checklist 

As  part  of  the  clinical  assessment  battery  in  our  trial,  patients  are  asked  to 
complete  the  Beck  Depression  Inventory-11  (BDI-II)  [24]  during  the  clinical 
outcomes  assessments  and  to  complete  the  Patient  Health  Questionnaire  (nine 
item)  [25]  before  each  treatment  session.  During  telehealth  sessions,  patients 
provide  their  responses  to  the  self-report  items  verbally  and  the  clinicians  record 
the  responses  in  the  patient’s  treatment  folder  (clinical  chart  notes).  These  assess¬ 
ment  measures  provide  a  method  for  regular  monitoring  of  clinical  symptom  levels 
and  the  presence  of  and/or  change  in  suicide  risk  throughout  treatment. 

For  all  patients,  regardless  of  their  initial  risk  level  determination,  ideation 
and  other  signs  of  risk  for  suicide  and  violence  are  documented  by  study  clinicians 
via  a  treatment  session  checklist.  Our  telehealth  treatment  session  checklist  was 
developed  in  part  to  provide  clinicians  with  specific  patient  safety  and  suicide  risk 
assessment,  monitoring,  and  documentation  procedures.  The  complete  checklist 
used  in  the  current  study  is  published  elsewhere  [16];  however,  its  components 
and  rationale  are  summarized  as  follows.  The  checklist  begins  with  a  verifica¬ 
tion  of  patient  location  and  contact  information.  Prior  to  the  first  session,  study 
clinicians  obtain  contact  information  for  local  emergency  services  based  on 
patient’s  place  of  residence.  Patients  are  also  asked  whether  they  feel  that  their 
home  environment  is  safe  and  private.  Additional  questions  assess  whether  the 
patient  appeared  intoxicated  or  otherwise  disheveled,  distressed,  or  upset;  suicidal 
desire  and  ideation,  such  as  whether  the  patient  showed  any  signs  of  suicidal 
ideation  or  self-harm  behavior;  and  plans  and  preparatory  behavior,  such  as 
whether  a  weapon  (firearm)  was  observed  in  the  patient’s  home. 

Risk  Escalation  and  Continuity  of  Care  Procedures 

For  the  purposes  of  our  trial,  if  a  study  clinician  observes  a  significant  elevation 
in  patient  risk  for  suicide  or  violent  behavior,  the  clinician  is  to  immediately 
notify  a  supervisory  psychologist  who  assists  in  determining  what  additional 
steps  of  the  safety  protocol  are  appropriate.  Although  determining  the  most 
appropriate  clinical  response  for  a  given  patient  requires  consideration  of  multiple 
factors,  our  safety  protocol  specifies  minimum  levels  of  intervention  to  be  offered 
at  each  risk  level.  For  example,  at  any  time  an  assessment  yields  a  risk  level 
determination  beyond  no  elevated  risk,  clinicians  are  expected  to  collaboratively 
develop  a  detailed  safety  plan  with  the  patient,  which  may  include  identification  of 
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safe  coping  strategies,  working  with  the  patient  to  remove  lethal  means  (e.g., 
storing  firearms  in  secured  locations)  and  involving  support  persons  with  the  plan. 

While  safety  plans  are  familiar  to  many  clinicians,  providing  patients  with  a 
copy  of  the  plan  during  telepractice  necessitates  additional  consideration.  In  our 
trial,  telehealth  patients  are  provided  with  blank  copies  of  our  safety  plan  at  their 
in-person  intake  assessments  so  that  the  plans  are  available  for  use  throughout 
treatment.  In  the  event  a  telehealth  patient  does  not  have  a  blank  copy  of  the  safety 
plan,  clinicians  collaborate  with  the  patients  via  telehealth  in  the  drafting  of  a 
safety  plan  and  then,  once  drafted,  review  the  contents  with  the  patient  and  allow 
the  patient  to  indicate  their  comfort  and  agreement  with  the  plan.  Clinicians  then 
mail  a  hard  copy  and/or  scan  and  e-mail  a  copy  of  the  plan  to  the  patient.  As  with 
conventional  in-office  care,  if  a  patient  is  assessed  to  be  at  high  risk  during  a 
telehealth  session,  clinicians  are  expected  to  consider  coordinating  an  evaluation 
of  the  patient  for  possible  hospitalization.  By  gathering  contact  information  for 
local  emergency  services  and  command,  and  identifying  an  emergency  contact 
on  behalf  of  the  patient  at  the  outset  of  telehealth  services,  our  safety  protocol 
is  designed  to  enable  clinicians  to  efficiently  and  effectively  coordinate  safe 
transport  and  evaluation  of  high  risk  patients. 

We  also  have  preplanned  procedures  in  place  to  assure  continuity  of  care  for 
when  patients  complete  treatment,  are  referred  but  do  not  enroll  in  treatment,  or 
if  they  leave  treatment  early.  To  begin,  we  work  with  each  patient  to  establish  a 
continuity  of  care  plan.  This  involves  discussing  with  patients  what  may  be  the 
best  options  for  them  given  their  preferences  and  clinical  needs.  We  also  facilitate 
coordination  with  the  initial  referring  care  providers  or  other  mental  health 
professionals  as  appropriate.  In  addition,  we  provide  all  of  our  patients  (regardless 
of  risk  level)  with  a  list  of  local  community  mental  health  resources  that  they 
may  keep  as  a  reference.  We  follow-up  with  patients  and/or  the  referring  care 
providers  when  necessary  and  document  as  appropriate.  Our  continuity  of  care 
process  helps  to  assure  patient  safety  after  they  leave  our  care. 

SUMMARY 

Our  safety  protocol  and  suicide  risk  assessment  procedures  include  gathering 
patient  information  so  that  we  can  make  informed  risk  assessments  and  enact 
indicated,  effective  responses  to  psychiatric  emergencies.  While  the  patients  in 
our  study  may  be  at  somewhat  lower  risk  for  suicide  than  other  clinical  patient 
populations  because  of  our  prescreening  criteria,  we  have  encountered  patients 
ranging  from  low  to  high  risk.  The  majority  of  our  patients  in  the  in-home 
treatment  condition  ( n  =  20  at  the  time  of  this  writing)  began  and  ended  treat¬ 
ment  identified  as  “not  at  elevated  risk”  for  suicide.  The  next  most  frequent 
risk  category  for  our  in-home  patients  has  been  low  (n  —  6  at  initial  treatment 
session).  In  each  of  these  cases,  the  patient  endorsed  a  history  of  suicidal  thinking 
with  limited  frequency,  intensity,  and  duration,  with  no  or  limited  plans  and  no 
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preparatory  behavior,  and  few  other  risk  factors  identified.  We  have  had  three 
patients  receiving  care  in  the  home  who  were  identified  as  intermediate  risk  for 
suicide  during  their  initial  risk  assessment.  For  each  of  these  cases,  the  steps  to 
take  in  the  event  of  increased  suicide  risk  were  discussed  with  the  patients.  We 
have  also  experienced  one  case  that  escalated  from  “low”  to  “high  risk”  and  one 
patient  that  was  assessed  to  be  “high  risk”  during  our  initial  assessment.  For  both 
of  these  cases  a  “warm  hand-off’  was  made  to  a  supervisory  clinician  for  further 
risk  assessment.  Given  the  high  level  of  risk  in  these  cases,  the  clinician  discussed 
voluntary  hospitalization  with  these  patients  and,  on  both  occasions,  the  patients 
agreed  that  presenting  to  inpatient  behavioral  health  for  evaluation  was  the  best 
option.  Per  U.S.  Army  regulations,  the  study  staff  contacted  the  Soldiers’  unit 
commanders  and  requested  escorts  to  transport  the  patients  to  the  ED.  One 
evaluation  resulted  in  inpatient  admission;  the  other  did  not,  although  it  did  result 
in  increased  intensity  of  care.  In  all  cases,  we  have  successfully  managed  suicide 
risk  by  following  pre-planned  recommended  procedures.  Our  work  demonstrates 
that  with  appropriate  planning  and  training,  patient  safety  can  be  effectively 
managed  during  telepractice,  even  when  patients  are  in  settings  that  are  not 
supervised  by  clinical  staff  (e.g.,  their  own  home). 

One  of  the  principal  concerns  regarding  safety  management  during  telehealth 
is  how  to  effectively  manage  a  suicidal  patient  when  the  telehealth  comiection  is 
lost  or  disconnected  during  a  clinical  encounter.  In  our  trial  there  have  not  been 
any  situations  where  technology  failures  caused  any  difficulties  with  risk  assess¬ 
ment,  monitoring,  or  intervention  procedures.  However,  consistent  with  practice 
recommendations  and  guidelines,  we  are  careful  to  collect  alternate  methods 
of  contact  in  case  the  videoconferencing  connection  is  lost.  We  also  identify  a 
support  person  who  can  assist  in  an  emergency.  It  is  also  important  to  tailor  safety 
plans  to  the  specific  situations  that  may  be  encountered,  particularly  if  patients 
are  located  in  another  geographical  or  jurisdictional  area.  Having  knowledge  of 
the  requirements  for  civil  commitment  and  Tarasoff  type  duty  to  wam/protect 
and  incorporating  these  elements  into  your  safety  plan  is  essential.  Even  at  a 
local  level,  simple  safety  procedures,  such  as  what  number  to  contact  for  emer¬ 
gency  response  services  may  vary  based  on  geographic  region. 

Suicide  risk  assessment  should  be  an  ongoing  process  as  risk  levels  are  fluid 
and  risk  determination  is  based  on  integration  of  multiple  pieces  of  information 
and  clinical  judgment  [2,  26],  While  assessment  guidelines  and  checklists  may 
enhance  the  standardization  of  the  assessment  process  and  reduce  errors,  suicide 
risk  is  conceptualized  as  existing  along  a  continuum  from  no  significant  risk  to 
imminent  risk  and  it  is  the  clinician  who  is  ultimately  charged  with  integrating 
present  and  historical  information,  considering  the  duration  and  severity  of 
explicit  and  implicit  risk  factors,  and  differentially  weighting  risk  and  protective 
factors  to  arrive  at  a  clinical  determination  [2,  26].  Maintaining  a  comprehen¬ 
sive  risk  management  or  safety  protocol  that  guides  the  assessment  process, 
encourages  consultation  with  colleagues  and  supervisors,  and  informs  clinical 
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decision  making  can  reduce  patient  and  clinician  anxiety,  enhance  accuracy  and 
reliability  of  the  assessment,  and  thereby,  supports  patient  safety  [27]. 

Safety  planning  with  patients  during  telepractice  may  also  carry  additional 
clinical  benefits  [28].  For  example,  the  process  of  working  collaboratively  with 
patients  to  establish  a  safety  plan  for  telehealth  encounters  may  provide  the  patient 
with  increased  confidence  and  therefore  contribute  to  improved  comfort  and 
acceptance  of  the  treatment  process.  Discussion  of  technical  procedures  as  well 
as  initial  testing  of  telehealth  equipment  may  also  help  to  facilitate  a  collaborative 
therapeutic  relationship.  In  some  cases,  the  involvement  of  a  family  member  or 
other  supportive  person  during  technical  set-up  and  as  part  of  safety  planning  may 
help  facilitate  patient  support  and  overall  treatment  adherence.  Of  course,  it  is 
necessary  to  consider  the  preferences  of  patients,  their  autonomy,  and  privacy 
risks  when  involving  others  in  their  care.  Telehealth  capabilities  also  provide 
increased  access  to  care,  especially  for  patients  who  reside  in  remote  or  under¬ 
served  areas.  For  these  patients,  access  to  care  via  telehealth  services  may  be 
critical  for  ongoing  treatment  of  suicidality,  including  assessment,  intervention, 
medication  management,  and  follow-up  care. 

In  conclusion,  telehealth  is  a  growing  area  of  practice  that  provides  oppor¬ 
tunities  to  increase  access  to  care,  improve  convenience,  and  expand  the  range 
of  clinical  services.  The  effective  assessment  and  management  of  patients  experi¬ 
encing  a  psychiatric  crisis  raises  important  legal,  operational,  and  clinical  issues 
that  telepractitioners  must  be  cognizant  of.  These  issues,  however,  should  not 
dissuade  practitioners  from  clinical  telepractice.  With  appropriate  safety  planning, 
training,  and  familiarity  with  published  guidelines,  telepractice  is  as  feasible  as 
traditional  in-office  clinical  care  delivery. 
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Purpose:  This  chapter  addresses  the  barriers,  solutions,  and  best  practices  for  conducting  clinical 
assessments  with  videoconferencing  technologies  (VCT).  Specific  attention  is  given  to  the 
process  of  determining  the  appropriateness  of  VCT-based  assessments  as  well  as 
recommendations  for  preparing,  optimizing,  and  conducting  clinical  assessments.  The  exchange 
of  assessment  materials,  data  security,  and  recommendations  for  assuring  the  integrity  of 
assessment  instruments  are  covered.  The  benefits  of  telehealth-based  assessments  and  future 
prospects  for  advancements  in  this  area  of  telebehavioral  health  practice  are  also  discussed. 
Context:  The  information  is  relevant  to  clinicians  and  clinical  managers  because  unfamiliarity  with 
practice  guidelines,  best  practices,  and  technical  capabilities  presents  a  critical  barrier  to 
competent ,  ethical,  and  effective  clinical  assessment  services. 

Tasks:  Specific  procedures  include  establishing  an  appropriate  setting  for  VCT  assessment,  conducting 
informed  consent  for  remote  services,  selecting  appropriate  measures,  managing  safety,  and 
providing  assessment  feedback. 

Tools:  The  chapter  provides  useful  information  for  selecting  compatible  assessment  measures  and 
procedures,  understanding  the  interaction  between  culture  and  service-delivery  via  VCT, 
practicing  within  legal  and  professional  boundaries,  and  developing  a  working  knowledge  of  the 
technological  aspects  of  VCT-based  clinical  assessments. 

Introduction 

This  chapter  addresses  best  practices  for  conducting  clinical  assessments  with  videoconferencing 
technologies  (VCT).  As  with  in-person  clinical  assessments  conducted  in  conventional  office  settings, 
telehealth-based  assessments  can  be  used  for  diagnostics,  screening,  symptom  monitoring,  evaluations 
of  treatment  progress  and  outcomes,  and  to  provide  further  understanding  of  client  contextual  factors 
and  needs.  Clinical  assessments  can  be  conducted  with  various  VCT  platforms  that  are  available  today 
including  professional  videoconferencing  systems,  webcams,  and  camera  enabled  mobile  devices  (i.e., 
smartphones  and  tablets  devices).  It  is  critical  for  clinicians  and  clinical  managers  to  be  familiar  with  the 
best-practices  for  VCT-based  assessments  in  order  to  assure  that  they  are  conducted  appropriately  and 
remain  clinically  useful.  Practitioners  also  need  to  know  whether  a  particular  measure  or  assessment 


technique  is  appropriate  for  use  with  VCT  and  they  need  to  become  familiar  with  the  proper 
administration  procedures  to  assure  competent  and  ethical  practice. 


The  chapter  begins  with  discussion  of  the  decision-making  process  regarding  whether  to  conduct  clinical 
assessments  remotely.  The  considerations  associated  with  clinical  appropriateness,  culture,  and 
acceptability,  legal,  regulatory,  safety  considerations,  and  practitioner  competencies  are  reviewed.  The 
issues  associated  with  selecting  appropriate  assessment  instruments  while  considering  the  evidence 
base  for  various  measures  and  techniques  are  discussed.  Best  practice  recommendations  for  preparing 
and  optimizing  assessment  conditions  and  the  procedures  for  conducting  assessments,  including 
methods  for  the  exchange  of  assessment  materials  are  discussed.  An  overview  of  technological 
considerations  along  with  data  security  and  best  practices  for  assuring  the  integrity  of  assessment 
instruments  is  also  provided.  The  chapter  concludes  with  a  discussion  of  the  benefits  of  telehealth- 
based  assessments  and  future  prospects  for  advancements  in  this  important  area  of  telebehavioral 
health  practice. 

Determining  the  Appropriateness  of  VCT-Based  Assessment 

The  appropriateness  of  VCT-based  clinical  assessment  must  be  determined  as  an  initial  step.  This 
involves  a  decision-making  process  that  begins  with  the  determination  of  the  objectives  for  the  clinical 
assessment  based  on  purpose,  setting,  and  clinical  needs  of  the  patient.  As  with  in-person  assessment, 
this  process  should  be  guided  by  evidence-based  practice  that,  as  described  by  the  American 
Psychological  Association,  involves  "the  integration  of  best  available  research  and  clinical  expertise  in 
the  context  of  patient  characteristics,  culture,  and  preferences."1 

While  the  decision-making  process  is  similar  to  clinical  assessment  conducted  in  conventional  office 
settings,  there  are  specific  issues  associated  with  telehealth-based  assessments  that  need  to  be 
considered.  The  patient's  attitudes  towards  and  experience  with  technology  are  important  factors,2  as 
are  the  clinician's  competency  and  level  of  comfort  with  conducting  clinical  assessments  via  telehealth 
technologies.3  There  are  also  aspects  of  the  specific  setting  where  services  are  to  be  provided  that  need 
to  be  evaluated.  These  include  considerations  regarding  safety  and  privacy  as  well  as  the  infrastructure 
and  the  technology  needed  to  conduct  VCT-based  assessments.  These  aspects  are  especially  critical 
when  providing  services  to  clinically  unsupervised  settings,  such  as  to  a  patient's  home.  This  is  because 
clinical  staff  cannot  directly  assist  with  setting  up  the  assessment  conditions  or  with  on-site  safety 
management  in  the  event  of  a  medical  emergency  or  clinical  crisis  (e.g.,  indication  of  imminent  intent  to 
harm  self  or  others).  In  these  settings,  the  clinician  must  work  with  patients  to  establish  appropriate 
assessment  conditions,  evaluate  potential  safety  risks,  and  establish  safety  procedures  such  as  the 
involvement  of  a  Patient  Support  Person  (PSP). 4-5 

In  each  of  the  following  sections,  the  issues  associated  with  determining  the  appropriateness  of  VCT- 
based  assessment  are  reviewed  in  greater  detail.  While  several  of  these  issues  are  discussed  in  other 
chapters  of  this  series,  they  are  presented  here  in  the  specific  context  of  VCT-based  clinical  assessment. 


It  is  important  to  note  that  while  these  issues  should  be  considered  prior  to  initiating  VCT-based  clinical 
assessments,  they  are  also  essential  to  incorporate  and  re-evaluate  throughout  the  assessment  process. 


Clinical  Appropriateness 

Patient  clinical  characteristics  (e.g.,  diagnosis,  cognitive  abilities,  etc.)  are  necessary  for  practitioners  to 
take  into  account  as  these  may  influence  whether  VCT-based  assessment  procedures  are  appropriate  or 
feasible.  For  example,  it  may  not  be  appropriate  to  conduct  an  assessment  with  VCT  with  a  person  who 
is  experiencing  delusional  beliefs  that  involve  technology  because  of  potential  iatrogenic  effects.  Clinical 
assessment  via  VCT  may  also  not  be  the  best  option  if  the  safety  of  the  patient  or  that  of  collaterals  (i.e. 
family  members  at  home)  is  a  concern  due  to  a  known  history  of  suicidal  or  violent  behavior.  It  is 
therefore  necessary  to  evaluate  and  assess  whether  VCT-based  assessments  are  appropriate  via  review 
of  treatment  records  or  other  patient  data  (if  available).  It  may  also  be  appropriate  to  consult  with  other 
practitioners  who  have  been  directly  involved  with  treatment  of  the  patient,  especially  in  clinical 
settings  where  the  person  conducting  the  assessment  is  someone  other  than  the  primary  care  provider. 
Suicide  risk  assessments  may  be  incorporated  into  prescreening  procedures  along  with  assessment  of 
other  patient  characteristics  such  as  vision  or  hearing  limitations. 

Clinical  contraindications  may  also  be  discovered  during  the  course  of  the  assessment  process.  For 
example,  it  may  be  discovered  that  a  patient  has  a  cognitive  deficit  or  problems  with  vision  not 
previously  identified  that  may  influence  the  validity  of  assessment  results.  In  those  cases,  clinicians 
should  consider  the  pros  and  cons  of  continuing  the  assessment,  and  as  with  any  conventional  in-person 
assessment,  document  and  disclose  how  those  factors  may  have  influenced  assessment  results. 

Culture  and  Acceptability 

Acceptability  of  assessment  procedures  can  influence  the  accuracy  of  any  assessment  as  well  as  a 
patient's  compliance  with  and  motivation  toward  the  demands  of  a  particular  assessment.6  Just  as  a 
patient's  cultural  background  is  relevant  to  the  procedures  and  interpretations  of  a  given  assessment 
tool, 7  a  patient's  culture  is  also  relevant  to  their  acceptability  and  level  of  comfort  with  remote 
assessment  and  the  VCT  medium.8 

Familiarity  and  comfort  with  technology  can  vary  based  on  individual  experiences  and  cultural 
backgrounds  and  these  variables  may  influence  patients'  acceptability  of  VCT-based  assessments. 
Familiarity  with  VCT  technology  has  become  more  commonplace  due  in  part  to  the  popularity  of  media 
services  and  videoconferencing  software  (e.g.,  Facetime,  Skype);  however,  significant  portions  of  the 
population  may  still  be  unfamiliar  with  these  technologies.  In  particular,  older  adults  represent  a 
growing  population  of  mental  health  consumers9  who  may  benefit  from  clinical  services  delivered  via 
VCT.10 11  This  population  may  be  less  familiar  with  VCT  technologies,  however,  and  they  may  prefer 
traditional  in-person  interactions  based  on  their  expectations  for  clinical  care  and  previous  experiences 
to  which  they  have  become  accustomed.12  Similarly,  expectations  and  accepted  norms  for  specific 
aspects  of  interpersonal  interactions  can  vary  considerably  in  different  cultures.  Another  example  is  the 


norm  of  eye  contact,  which  may  vary  across  cultures.  In  Native  cultures,  for  example,  direct  eye  contact 
may  be  considered  disrespectful  and  rude.13  Thus,  clinicians  need  to  consider  cultural  expectations 
associated  with  eye  gaze  when  using  VCT  technologies  as  this  is  important  for  both  establishing  rapport, 
and  preventing  misinterpretation  of  behaviors  during  a  clinical  assessment  (e.g.,  mistaking  a  consistently 
downward  eye  gaze  as  a  symptom  of  depression). 

As  noted  by  Luxton  and  colleagues,14  technical  difficulties,  especially  those  that  interrupt  the  interaction 
between  the  patient  and  care  provider,  can  reduce  satisfaction  with  VCT-based  services.  Problems  with 
network  connectivity,  low  bandwidth  speed,  sound  echo/feedback,  and  an  inability  to  transmit  written 
material  (e.g.,  self  report  questionnaires)  between  users  have  been  specifically  identified  as  sources  of 
frustration.15'16  The  loss  or  distortion  of  non-verbal  behavior  and  other  patient  characteristics  may  also 
negatively  impact  the  acceptability  of  TMH  assessment.17  However,  solutions  to  these  issues  are 
increasingly  available  (e.g.,  faster  internet  speeds/video  compression,  head  sets,  adjunct  technology 
such  as  fax/document  scanners,  software  that  allows  screen  sharing)  but  come  with  financial  costs  that 
may  not  always  be  practical.  Assessment  of  potential  technical  problems  and  limitations  (e.g.,  network 
bandwidth,  etc.)  is  recommended  prior  to  initiating  VCT-based  assessments. 

Also  important  for  the  initial  decision  making  process  is  the  practitioner's  own  comfort  level  and 
experience  with  technology  and  its  use  during  clinical  assessments.  While  it  is  likely  not  necessary  for 
the  clinician  to  be  an  expert  regarding  how  the  technology  works,  it  is  important  for  the  clinician  to  be 
adequately  competent  with  using  it  and  confident  that  he  or  she  can  effectively  troubleshoot  technical 
issues  if  they  occur  during  an  assessment  session.  Access  to  troubleshooting  guides  and  a  plan  to  involve 
external  technical  support  or  in  some  cases  a  Patient  Support  Person  (PSP)  to  assist  with  technical  issues 
is  recommended  and  should  be  considered  before  initiating  VCT-based  assessments. 

Clinicians  and  clinical  managers  must  remain  mindful  of  the  cultural  factors,  patient  comfort  with 
technology,  and  both  patient  and  provider  familiarity  with  the  technical  aspects  of  remote  service 
delivery.  Clinicians  should  also  provide  a  clear  rationale  to  patients  for  why  VCT  is  being  used.  This 
rationale  should  include  discussion  of  the  benefits  that  VCT-based  clinical  services  can  offer  as  well  as 
limitations,  risks  (e.g.,  security  of  data  transmissions),  and  any  potential  difficulties  that  may  be 
encountered  when  using  the  VCT  medium.18 

Modai  and  colleagues  reported  that  there  is  a  positive  correlation  between  a  person's  regular  use  of  VCT 
and  their  acceptability  of  it,  suggesting  that  experience  using  VCT  in  clinical  practice  may  strengthen  the 
user's  opinion  of  that  modality.19  Moreover,  the  perceived  expertise  of  the  clinician  as  well  as  the 
distance  required  to  receive  an  in-person  alternative  (i.e.,  travel  burden)  may  also  influence 
acceptability.20  Fortunately,  across  diverse  clinical  populations  and  services  there  are  consistently 
positive  ratings  of  user  satisfaction  and  acceptance  with  VCT,  which  are  either  equivalent  or  in  some 
cases  superior  to  ratings  of  in-person  services.10, 11 


Legal,  Regulatory,  and  Safety  Considerations 


Licensing  and  legal  issues  also  have  great  bearing  on  the  decision  to  conduct  clinical  assessments 
remotely.  State  laws  specific  to  the  practice  of  telemedicine  may  vary  greatly,  and  many  jurisdictions 
require  practitioners  to  be  licensed  in  the  same  state  in  which  the  patient  is  located  for  treatment.21'22 
This  has  the  potential  to  limit  delivery  of  psychological  assessment  services  depending  on  practitioner 
license  and  client  location,  with  the  exceptions  of  practitioners  working  within  a  federal  jurisdiction  (e.g. 
VA  medical  centers)  who  may  be  able  to  provide  telehealth  services  across  state  lines  to  other  federal 
sites.  These  issues  become  increasingly  challenging  in  parallel  with  the  complexity  of  a  patient's 
situation.  For  example,  a  patient  in  treatment  at  home  may  be  travelling  to  another  state  for  business  or 
vacation  during  a  scheduled  follow-up  assessment.  Thus,  it  is  essential  that  clinicians  verify  where  the 
patient  is  physically  located  and  become  familiar  with  both  the  laws  governing  the  jurisdiction  in  which 
they  plan  to  conduct  assessments  and  the  jurisdiction  in  which  their  clients  are  located. 

Mandatory  reporting  and  involuntary  commitment  laws,  for  example,  can  differ  between  states  and 
practitioners  should  follow  the  laws  established  by  the  jurisdiction  in  which  the  patient  is  physically 
located  when  carrying  out  legal  duties  and  safety  planning.22  There  may  also  be  laws  specific  to 
assessments  conducted  remotely  and  regulations  governing  delivery  of  results.  For  example,  federal 
legal  precedent  has  established  VCT  as  an  acceptable  means  of  suicide  risk  assessment  in  involuntary 
commitment  contexts,23  although  states  may  establish  more  stringent  protections  than  federal  law,  and 
some  states  have  granted  citizens  the  right  to  appear  in  person  instead  of  via  VCT.24  Given  the  remote 
nature  of  VCT  interventions  and  the  increased  number  of  clinically  relevant  factors  involved,  safety 
considerations  should  be  a  priority  throughout  the  assessment  process.  In  order  to  manage  risk 
effectively,  ensure  patient  safety,  and  comply  with  legal  responsibilities,  practitioners  must  integrate  a 
working  knowledge  of  practical  legal  issues  into  their  decision-making  along  with  consideration  of  client 
background  and  contextual  factors. 

Clinician  Competencies 

The  practice  of  TMFI  involves  specialized  knowledge,  skills,  and  professional  standards.25'26  Clinicians 
must  assess  their  own  level  of  skill  and  competence  as  part  of  the  process  of  deciding  whether  they 
should  conduct  telehealth-based  assessments.  Competencies  include  specific  knowledge  and  skills 
required  to  administer  assessment  measures  and  tests  via  VCT,  familiarity  with  clinical  best  practices  for 
conducting  TMFI  sessions,  safety  management,  as  well  as  legal  and  ethical  standards.  Clinicians  must 
also  be  technically  proficient  with  the  specific  equipment,  software,  and  network  connection  to  be  used 
during  the  assessment  sessions  in  order  to  minimize  potential  technology  problems.5, 27  This  requires  a 
working  knowledge  of  clinical  guidelines  for  practical  aspects  of  using  VCT  equipment  (e.g.,  bandwidth, 
camera  position)  as  well  as  clinical  skills  relevant  to  VCT-based  assessment  such  as  administration 
protocols  for  remote  assessment. 

Clinicians  and  clinician  managers  should  become  familiar  with  published  practice  guidelines  by 
professional  organizations  such  as  the  American  Psychological  Association28  and  the  American 
Telemedicine  Association.29, 4  Additional  training  in  clinical  telepractice  and  review  of  relevant  scientific 
literature  are  also  important  aspects  of  maintaining  culturally  and  clinically  competent  practice. 


Although  some  continuing  education  is  currently  available  through  several  for  profit  and  nonprofit 
organizations  (e.g.,  American  Psychological  Association,  American  Telemedicine  Association), 
researchers  have  proposed  more  formalized  training  models  for  use  in  graduate  programs  and 
professional  workplace  settings.  For  example,  Godleski  has  proposed  training  programs  for  TMH 
psychiatric  services30  and  Colbow  presented  curriculum  content  with  a  component  involving  supervised 
experience  with  TMH  technologies.25  Clinicians  should  also  develop  the  necessary  basic  knowledge, 
skills,  and  experience  required  to  effectively  use  and  troubleshoot  VCT  equipment. 

Selecting  Appropriate  Measures  and  Procedures 

Just  as  with  clinical  assessments  made  in  conventional  in-person  settings,  practitioners  must  consider 
whether  the  measures  and  procedures  for  the  administration  and  scoring  of  assessment  instruments  are 
appropriate  for  the  specific  application  and  patient  population.  This  includes  consideration  of  available 
validity  and  reliability  data  regarding  specific  measures  and  procedures.  Luxton  and  colleagues  noted 
that  just  because  the  validity  and  reliability  of  a  particular  instrument  or  procedure  has  been  established 
for  in-person  administration,  it  does  not  mean  that  those  qualities  of  an  assessment  will  hold  when  it  is 
administered  over  VCT.3  The  decision  to  administer  any  particular  measure  via  VCT  must  therefore  be 
based  on  available  psychometric  data  as  well  as  information  regarding  procedures  for  modifying 
measures  and  procedures  for  remote  administration. 

Several  available  published  reviews  provide  useful  information  regarding  the  validity  and  reliability  of 
clinical  assessments  conducted  via  telehealth  technologies.  For  example,  Hyler  and  colleagues 
conducted  a  review  and  meta-analysis  of  14  studies  that  compared  face-to-face  to  TMH  psychiatric 
assessments.31  Five  of  the  studies  used  objective  assessment  measures  (e.g.,  clinical  rating  scales),  two 
studies  used  subjective  measures  (i.e.,  satisfaction  measures),  and  seven  studies  used  the  combination 
of  objective  and  subjective  measures.  The  meta-analytic  results  indicated  that  objective  assessments 
were  comparable  to  in-person  assessments  in  regard  to  diagnosis  or  symptoms  assessment.  The 
Telemental  Health  Standards  and  Guidelines  Working  Group32  also  reviewed  and  reported  data 
regarding  the  psychometric  properties  of  remote  telehealth-based  assessments.  These  authors 
highlighted  evidence  that  supported  the  use  of  standard  objective  measures  via  TMH,  such  as  the 
Hamilton  Depression  Rating  Scale,  with  some  caveats  about  the  negative  effects  of  low  bandwidth  on 
reliability  for  symptom  rating  scales.33"34  They  also  noted  support  for  the  reliability  of  structured  clinical 
interviews.20, 35  A  later  systematic  review  by  Backhaus  and  colleagues10  supported  these  conclusions  and 
showed  that  the  majority  of  studies  they  evaluated  (69%)  utilized  "at  least  one  standardized  measure 
with  well-accepted  psychometrics"  (p.  118),  including  rating  scales  such  as  the  BDI-II36  and  the 
Structured  Clinical  Interview  for  the  DSM-IV  Axis  I  Disorders,  Research  Version,  Patient  Edition  (SCID- 
l/P).37 

The  BDI-II36  is  a  commonly  used  measure  reported  in  published  TMH  treatment  research10  that  has 
evidence  for  its  comparability  in  computer  and  paper  forms.38  Evidence  of  the  reliability  and 
comparability  to  in-person  assessments  for  structured  clinical  interviews  and  psychiatric  evaluations 
conducted  via  VCT  have  also  been  reported.  35  39  Cognitive  measures  have  been  administered  via  VCT  as 


well,  and  patients  have  generally  indicated  acceptance  of  and  satisfaction  with  the  assessment 
process.40'41  A  small  number  of  studies  have  investigated  the  agreement  between  neuropsychological 
tests  conducted  in-person  and  via  VCT40'41  and  generally  found  adequate  comparability,  with  some 
evidence  for  greater  reliability  with  measures  that  primarily  use  verbal  instructions.41 

Researchers  have  highlighted  a  gap  in  the  literature  base  regarding  VCT-based  assessments.25  Some 
have  pointed  out  the  need  to  establish  separate  psychometric  norms  26  whereas  others  have 
recommended  the  develop  of  assessments  specifically  designed  for  use  with  TMH  technologies.  10 
Special  cautions  for  neuropsychological  measures  have  been  suggested  given  the  underdeveloped 
protocols  for  adapting  established  measures  as  well  as  the  paucity  of  research  on  delivery  through  VCT, 
which  may  influence  testing  procedures,  accuracy  of  assessments,  and  interpretation  of  results.26,42  For 
example,  the  established  normative  performance  data  for  the  Halstead-Reitan  Neuropsychological 
Battery  were  developed  well  before  the  advent  of  the  personal  computer,  thus,  it  may  be  inappropriate 
to  use  these  data  to  gauge  performance  on  tests  if  administered  remotely  via  VCT.  Colbow  suggested 
that  similar  caution  be  applied  to  cognitive  testing  and  personality  assessments  due  to  the  lack  of  data 
regarding  the  use  of  standardized  measures  that  are  administered  electronically.25  Grady  and  colleagues 
noted  that  research  on  personality  assessment  delivered  via  VCT  is  nonexistent  and  that  "there  is  no 
information  about  projective  testing  over  VCT  (p.  138)"  while  neuropsychological  testing  is  feasible  but 
"may  lack  the  same  degree  of  resolution  that  [face-to-face]  assessment  provides"  (p.  138). 32 

It  is  important  to  note  that  assessment  objectives  and  the  client's  needs  may  require  a  particular  type  of 
assessment  that  lacks  an  established  evidence-base  for  the  use  of  VCT  administration  or  that  is  not 
currently  feasible  for  remote  administration.40, 3  Thus,  clinicians  should  consider  the  available 
alternatives  for  clinical  assessment  and  what  may  best  meet  the  needs  of  the  particular  situation.  In 
some  cases,  remote  assessment  via  VCT  may  be  the  only  feasible  method  and  therefore  the  best 
available  option  to  a  particular  patient.  This  may  be  true  for  populations  that  live  in  remote  areas,  such 
as  parts  of  Alaska,  which  may  become  inaccessible  or  very  treacherous  to  travel  through  during 
substantial  portions  of  the  year.  A  home-bound  elder  that  risks  a  life  threatening  fall  by  traveling  to  a 
clinic  is  another  example  of  a  situation  in  which  VCT  assessments  may  be  the  best  choice. 

In  summary,  clinicians  who  are  interested  in  conducting  assessment  via  VCT  must  consider  the  evidence 
base  supporting  specific  measures  and  their  use  with  this  medium.  Recommendations  for  appropriate 
adaptation  of  measures  and  administration  procedures  should  also  be  considered.  Recent  research 
reviews  support  the  use  of  many  popular  clinical  rating  scales,  structured  interviews,  neuropsychological 
measures,  and  satisfaction  measures  administered  via  VCT.  There  are,  however,  limitations  to  the 
available  data  that  will  require  additional  study.  The  need  for  separate  norms  for  existing  tests  as  well  as 
measures  designed  specifically  for  TMH  administration  has  been  recommended. 

Preparing  and  Conducting  Assessments 


Informed  Consent  Process 


Behavioral  health  clinicians  who  intend  to  provide  VCT-based  assessment  services  must  consider  the 
requirements  of  informed  consent.  While  most  of  the  informed  consent  components  are  consistent  with 
the  requirements  for  in-person  mental  health  services,  there  are  also  unique  aspects  associated  with  not 
being  in  the  same  room  as  the  patent  as  well  as  the  use  of  technology.  An  initial  consideration  is  the 
need  to  confirm  the  patient's  identity  remotely  in  order  to  assure  patient  privacy  and  to  prevent  service 
delivery  to  non-patients.4  Unless  the  clinician  and  patient  have  previously  met  in-person,  presentation  of 
supporting  documents  (e.g.,  driver's  license,  insurance  information,  etc.)  via  VCT  may  be  necessary. 

Once  initial  positive  identification  has  been  made,  visual  confirmation  of  patient  identity  during 
subsequent  VCT  sessions  should  be  adequate.  The  issue  of  positive  identification  may  be  complicated, 
however,  when  only  audio  (e.g.,  telephone)  is  available.  It  is  therefore  important  to  discuss  expectations 
for  establishing  identity  during  the  informed  consent  process.  Also,  as  with  any  clinical  service, 
practitioners  should  clearly  identify  themselves  and  their  credentials  during  the  informed  consent 
process.4  The  process  for  doing  this  may  be  accomplished  with  technological  solutions  such  as  displaying 
credentials  over  VCT  or  electronically  on  a  website.  The  requirement  for  assuring  patient  identity  and 
communicating  credentials  may  vary  by  jurisdiction  and  thus  clinicians  and  clinician  managers  should 
become  familiar  with  applicable  laws  and  guidelines. 

The  informed  consent  process  should  also  include  a  discussion  of  the  potential  risks  and  relevant  issues 
for  the  modality  through  which  those  services  will  be  delivered.4, 28, 43  In  regards  to  VCT-based  clinical 
assessments,  clinicians  should  discuss  the  ways  that  the  modality  may  differ  from  alternative  options  as 
well  as  the  unique  risks  and  benefits.28  Of  particular  relevance  is  disclosure  of  confidentiality  and  privacy 
measures  and  expectations  associated  with  the  electronic  storage  and  transmission  of  assessment  data 
and  results.  Practitioners  should  also  discuss  with  the  patient  that  there  is  a  possibility  that  the  accuracy 
of  the  assessment  data  collected  could  be  influenced  by  the  quality  of  the  VCT  connection  or  other 
disruptions.  This  is  especially  important  in  circumstances  where  assessment  performance  will  be  used  to 
make  lasting  decisions  about  a  patient's  care,  entitled  benefits,  or  legal  status. 

In  summary,  there  are  several  unique  aspects  of  telehealth-based  clinical  assessment  that  clinicians 
must  consider  during  the  informed  consent  process.  Clinicians  must  also  be  sure  to  provide  ample  time 
to  address  any  questions  that  the  patient  may  have  regarding  VCT-based  clinical  assessment.  Given  the 
evolving  laws  and  guidelines,  practitioners  must  also  remain  current  on  laws  and  practice  standards  that 
have  impact  on  the  overall  informed  consent  process.4'28 

Optimizing  Assessment  Conditions 

When  preparing  for  a  remote  assessment,  the  practitioner  must  prepare,  or  oversee  preparation  of,  the 
rooms  at  both  ends  of  the  connection.  In  addition  to  assuring  that  all  assessment  materials,  scoring 
materials,  measurement,  and  timekeeping  tools  are  in  place,  the  clinician  must  also  verify  that  their  VCT 
equipment  is  capable  of  meeting  minimal  technical  standards  (e.g.,  appropriate  bandwidth  speeds)  and 
is  readied  for  use  (e.g.,  microphone  and  speakers  volume  set,  camera  lens  cap  removed,  camera 
properly  positioned,  etc.).  Practitioners  should  also  assure  that  that  their  clinical  space  is  adequately  lit 
and  that  the  background  provides  adequate  image  clarity.44 


The  room  at  the  patient's  end  of  the  connection  must  also  be  prepared.  In  situations  where  VCT  services 
are  provided  to  a  remote  clinic  space,  support  staff  may  assist  with  preparation  of  the  assessment 
space.  This  includes  making  sure  the  room  is  private  and  comfortable  and  assuring  that  the  assessment 
materials  (i.e.,  writing  instruments,  forms,  and  questionnaires)  are  available  and  prepared.  When  clinical 
support  staff  is  unavailable,  such  as  during  home-based  assessments,  the  patient  may  be  asked  to 
prepare  the  room  or  ask  a  PSP  for  assistance.  For  some  types  of  clinical  assessment,  clinicians  may  also 
need  to  mail  assessment  materials  to  the  remote  location  ahead  of  time.  Clinicians  must  verify  that 
patients  have  adequate  materials  and  space  available  to  complete  the  assessment  (e.g.,  writing  tools, 
desk  space,  etc.). 

One  of  the  primary  factors  that  may  influence  the  validity  and  reliability  of  remote  assessments 
conducted  via  VCT  is  the  fact  that  the  client  and  practitioner  are  not  physically  in  the  same  room.3  This 
has  obvious  bearing  on  procedures  for  administering  assessments  as  well  as  what  information  can  be 
assessed  during  clinical  assessment  procedures.  In  particular,  the  lack  of  in-person  presence  can  limit 
what  information  can  be  assessed.  Often,  a  patient's  nonverbal  cues  provide  useful  information  about 
their  emotional  state,  symptom  severity,  and  even  risk  behaviors.  Olfactory  data  can  provide  relevant 
information  regarding  a  patient's  alcohol  and  substance  use,  as  well  as  hygiene.  Clinical  impressions  and 
conceptualizations  can  be  influenced  by  body  posture  and  language  (e.g.,  foot  tapping,  hand  wringing), 
minute  changes  to  facial  expression,  and  other  non-verbal  emotional  responses  such  as  facial  flushing, 
tearing  up,  and  eye-gaze  direction,  which  may  be  absent  during  VCT  assessment  depending  on  the 
camera  position  and  resolution  as  well  as  the  quality  of  the  transmission.  For  example,  when 
administering  the  Brief  Psychiatric  Rating  Scale  (BPRS),45  the  practitioner  may  need  to  ask  additional 
questions  regarding  the  somatic  manifestation  of  symptoms  in  order  to  most  accurately  determine  the 
rating  that  best  describes  a  patient's  symptoms  for  domains  such  as  anxiety,  tension,  mannerisms  and 
posturing,  motor  retardation,  and  excitement  since  observation  of  the  patient's  full  body  may  not  be 
possible. 

Psychomotor  and  other  medically  relevant  symptoms  may  need  to  be  observed  during  psychological 
assessments  as  well,  in  which  case  the  practitioner  can  query  them  directly  (e.g.,  to  assess  gait 
problems,  the  patient  may  need  to  be  asked  to  stand  back  from  the  camera  and  walk  across  the  room). 
Furthermore,  the  observation  of  how  an  examinee  approaches  a  test  or  measure  may  be  critical  for 
making  an  accurate  assessment.  For  example,  along  with  measuring  the  accuracy  of  an  examinee's  copy 
and  recall  drawings  of  the  Rey-Osterrieth  Complex  Figure,46  it  is  important  to  examine  the  process  by 
which  a  patient  completes  their  drawing.  With  a  camera  capturing  only  a  patient's  face,  the  provider 
may  not  have  access  to  this  information.  In  this  situation,  and  other  similar  situations,  the  remote 
location  may  need  multiple  cameras  so  that  the  practitioner  is  able  to  view  the  patient's  desktop  surface 
where  they  are  completing  assessment  tasks. 

In  some  cases,  in-person  assessment  procedures  may  need  to  be  modified  or  adapted  to  allow  for 
accurate  VCT  assessment.  Practitioners  are  cautioned,  however,  to  carefully  review  the  instructions  or 
administration  manuals  of  the  planned  measures  and  tests  to  ensure  that  the  appropriate  procedures 


and  environmental  conditions  for  standardized  administration  are  maintained  in  a  way  that  does  not 
threaten  the  reliability  or  validity  of  a  given  assessment.  Rather  than  viewing  a  patient's  responses 
across  a  table,  it  may  be  necessary  to  ask  a  patient  to  hold  their  responses  to  a  paper  and  pencil 
assessment  up  to  the  video  camera  for  viewing  (e.g.,  self  report  measures).  Patients  may  also  need  to  be 
asked  to  use  larger  handwriting  if  small  screen  sizes  or  poor  image  quality  disrupt  the  provider's  ability 
to  view  their  responses.  Alternatively,  it  may  be  helpful  to  ask  the  patient  to  read  their  responses  out 
loud  in  scenarios  where  video  is  not  used  or  if  the  connection  quality  is  poor.  Practitioners  should 
monitor  their  pace,  annunciation,  and  volume  of  speech  to  ensure  that  patients  receive  the  directions 
clearly.  Patients  should  be  asked  to  paraphrase  instructions  back  to  the  practitioner  to  ensure  clear 
receipt  of  a  task's  demands.  While  it  is  advisable  to  do  this  regardless  of  assessment  medium,  it  may  be 
particularly  important  to  do  so  when  making  use  of  VCT  as  occasional  fluctuation  in  bandwidth  may 
briefly  distort  audio  transmissions. 

Technology  Specific  Considerations 

There  are  several  factors  associated  with  the  use  of  technology  that  may  influence  the  reliability  and 
validity  of  clinical  assessment  when  videoconferencing.  Eye  gaze  angle,  the  angle  between  the  eye  and 
the  camera  and  the  eye  and  the  center  of  the  display,  is  one  issue  that  has  been  discussed  depth.47 
Users  of  VCT  tend  to  make  eye  contact  with  the  image  on  the  screen  rather  than  with  the  camera.48  This 
tendency  may  lead  the  person  on  the  other  end  of  the  connection  to  the  impression  that  the  person  is 
avoiding  eye  contact.  Eye  contact  between  a  patient  and  clinician  is  important  because  it  provides  visual 
cues  to  which  the  participants  can  respond  and  is  also  a  marker  for  interpersonal  skill  and  social 
ability.47,49  Eye  contact  is  also  an  important  source  of  clinical  information  for  determining  the  presence 
of  psychological  states  (e.g.  delirium)  or  particular  disorders  (e.g.,  Autism  Spectrum  Disorder).  When  eye 
contact  is  ambiguous,  interpretation  of  facial  expressions  and  affect  may  become  difficult.  Eye  gaze 
angle  may  also  influence  satisfaction  with  using  VCT.47  Cameras  should  be  positioned  in  a  way  that 
allows  the  images  of  both  parties  to  appear  straight-on  and  centered  in  their  respective  monitors  so  that 
both  appear  to  speak  eye-to-eye  with  each  other.44  Additionally,  improved  eye  contact  can  be  realized 
by  increasing  the  horizontal  distance  of  participants  from  the  videoconferencing  unit.47 

Patients  may  shift  position  or  the  video  camera  may  accidently  be  shifted  from  the  optimal  angle  during 
the  course  of  an  assessment.  It  may  therefore  be  necessary  to  make  adjustments  to  the  camera  position 
or  to  the  patient's  physical  location  in  relation  to  the  camera.  This  can  be  accomplished  by  asking  the 
patient  to  make  adjustments  to  camera  angle  or  their  physical  position.  Some  VCT  camera  systems 
include  a  remote  tilt-and-zoom  and  panning  feature  that  may  make  sense  to  use.  It  is  also 
recommended  to  use  the  'picture  in  picture'  function,  available  on  many  VCT  devices,  to  provide  the 
clinician  with  visual  verification  of  proper  framing.  As  noted  by  Luxton  and  colleagues,3  it  is  also  advised 
to  check-in  with  patients  throughout  the  assessment  process  to  make  sure  that  they  can  see  and  hear 
clearly.  It  is  important  to  remember  that  technological  limitations  may  influence  how  well  the  patient 
understands  the  practitioner  (not  just  how  well  the  practitioner  understands  the  patient). 


Audio  quality  is  another  important  factor.  Low  microphone  input  volume  or  speaker  output  volume  may 
make  it  difficult  to  understand  questions,  responses,  and  instructions.  Jones  and  colleagues50  found  that 
inadequate  audio  quality  can  influence  the  ability  to  accurately  gather  information  from  the  patient.  For 
example,  the  observation  of  vocal  properties  (e.g.,  shakiness,  inflection,  and  tone),  as  well  as  whether  an 
individual  may  be  crying,  can  be  an  important  source  of  information  regarding  emotional  states,  and  a 
low-quality  audio  connection  may  hamper  observation  of  this  information.  Volume  levels  should 
therefore  be  established  and  tested  at  the  outset  of  the  interaction,  and  must  be  weighed  against  the 
possibility  that  the  audio  could  be  overheard  by  others.  As  with  conventional  office  settings,  the  use  of  a 
white-noise  machine  outside  the  assessment  room  door  may  make  sense. 

Additionally,  the  transmission  lag  (the  time  it  takes  for  the  audio  or  video  signal  to  travel  from  one  user 
to  the  other)  may  influence  the  natural  flow  of  conversation  during  assessments.  A  longer-than-normal 
pause  after  asking  a  question  before  the  patient  answers  can  be  expected  in  some  cases.  Each  speaker 
may  need  to  pause  every  few  sentences  to  allow  enough  time  for  the  other  person  to  respond.  Each 
should  speak  at  a  slightly  slower  pace  so  that  the  normal  indications  from  the  listener  (e.g.  a  brief 
comment  or  raised  hand)  can  be  transmitted  and  received  before  the  speaker  continues  on  to  their  next 
statement. 

Even  if  the  camera  is  positioned  perfectly  and  audio  quality  is  just  right,  a  poor  network  connection  can 
still  interfere  with  the  capabilities  and  quality  of  an  assessment.  A  variety  of  factors  can  contribute  to 
network  connection  problems,  including  low  quality  equipment,  an  overloaded  computer  (e.g.,  too 
many  programs  running  at  one  time),  inadequate  bandwidth,  and  user  inexperience  with  VCT.  '  '  For 
example,  Hyler  and  colleagues  found  that  both  patients  and  practitioners  reported  a  preference  for  in- 
person  assessment  when  the  available  VCT  was  hampered  by  low-bandwidth.31  Interestingly,  when  high- 
bandwidth  VCT  was  available  it  was  preferred  over  in-person  assessment,  even  when  the  assessments 
being  employed  required  detailed  observations  of  the  patient's  behavior.  Additionally,  Zarate  and 
colleagues  noted  that  both  patients  and  practitioners  found  high  and  low  quality  VCT  acceptable,  but 
that  higher  quality  VCT  resulted  in  a  greater  degree  of  reliability  between  assessments.51  Although  there 
is  some  guidance  available  regarding  minimum  recommended  bandwidth  requirements  for  TMFI  (e.g., 
384  Kbps  for  downlink  and  uplink4' 29),  what  is  or  is  not  adequate  in  any  given  application  will  depend  on 
a  variety  of  factors,  including  requirements  for  the  type  of  assessment,  environmental  conditions,  and 
the  technology  itself. 

It  is  also  possible  that  the  use  of  technology  (e.g.,  web  cam,  personal  computer,  microphone)  may 
distract  or  require  extra  attention  that  may  influence  the  assessment  session.  For  example,  when 
conducting  a  clinical  interview  over  a  web  cam,  the  patient  may  become  distracted  by  inconsistent 
connections,  error  messages,  or  other  technical  anomalies.52  Shifts  in  a  patient's  attention  may  influence 
testing  or  assessment  scores,  potentially  decreasing  performance  in  domains  requiring  sustained 
attention  and  concentration.  Additionally,  lengthy  or  complex  assessment  procedures  can  be  quite 
demanding  on  patients.  While  longer  and  more  in-depth  assessments  generally  provide  the  greatest 
reliability,  participant  dropout  is  also  much  more  likely.53  Overly  long  assessments  may  not  be  suited  for 
VCT  assessment.54  Exposure  to  a  new  medium  of  interaction  (VCT)  for  persons  who  are  not  experienced 


with  it  may  add  additional  stress  or  frustration,  especially  if  technical  problems  occur.15 16  Given  this, 
clinicians  must  remain  mindful  of  the  patient's  degree  of  frustration  and  how  likely  it  is  that  providing  a 
given  assessment  in  a  novel  manner  may  affect  a  patient's  performance  or  patterns  of  responding. 


Despite  the  many  factors  that  can  influence  VCT  transmissions,  efforts  should  be  made  to  maximize 
quality.  While  it  may  seem  difficult  initially  to  manage  all  of  the  various  components,  with  practice  the 
preparation  necessary  for  a  high-quality  VCT  interaction  can  become  seamless  and  routine.  It  is 
important  for  clinicians  and  clinician  managers  who  plan  to  use  VCT  to  obtain  training  and  experience 
with  the  technology  prior  to  offering  these  services.  Practice  sessions  are  highly  encouraged,  including 
the  simulation  of  common  technical  difficulties. 

Privacy  and  Assessment  Data  Security 

Risks  to  confidentiality  can  occur  at  both  the  patient  and  clinician  ends  of  the  VCT  transmission. 
Precautions  should  therefore  be  taken  to  assure  that  the  environments  at  both  ends  are  private  and 
secure  during  assessment  sessions.  Doors,  windows,  and  window  coverings  should  be  secured  to 
prevent  audio  and  visual  content  from  traveling  outside  of  the  assessment  space.  VCT  equipment 
volume  should  be  adjusted  to  a  level  where  the  patient  can  adequately  hear  the  practitioner  (and  vice 
versa)  but  is  not  so  loud  that  others  outside  of  the  room  can  overhear  the  interaction.  For  in-home 
telehealth-based  clinical  assessments,  the  patient  may  need  to  negotiate  with  their  family  members  or 
roommates  for  private  time  and  space  for  assessment  sessions.  As  noted  earlier  in  this  chapter,  patients 
may  also  be  concerned  about  the  security  of  the  clinician's  environment,  such  as  whether  the 
interaction  is  being  recorded  and  who  has  access  to  it,  and  if  anyone  other  than  the  practitioner  is 
present  in  the  room  "off  camera."  Clinicians  should  assure  adequate  time  and  opportunity  for  patients 
to  ask  questions  and  to  address  any  concerns  that  the  patient  may  have  about  the  process. 

Some  have  suggested  that  allowing  the  patient  some  time  to  get  accustomed  to  this  new  medium  is 
beneficial,  which  may  include  taking  physical  steps  to  aid  in  this  process  (e.g.  having  the  practitioner  use 
the  camera  to  tour  them  through  their  office).26  Privacy  and  data  security  are  also  important  aspects  of  a 
TMH  informed  consent  process,  as  special  attention  must  be  given  to  privacy  and  the  limits  of 
confidentiality  in  TMH  contexts  given  the  transmission  of  VCT  sessions  over  some  type  of  network26 
which  always  carries  some  security  risk.  In  addition,  questions  about  the  safety  and  storage  of 
assessment  documents  and  electronic  records  can  be  discussed  as  part  of  a  comprehensive  informed 
consent  process.26 

Safety  Planning 

Given  the  unique  factors  involved  in  TMH  assessment,  careful  consideration  of  patient  safety  and 
planning  for  risk  management  procedures  must  be  in  place  when  delivering  VCT  services,  including 
clinical  assessments.  Practitioners  must  prepare  the  necessary  resources  and  establish  adequate  means 
for  managing  a  patient's  distress  and  risk  for  self-harm  or  violence  during  TMH  assessment,  and  all  of 
these  procedures  should  be  discussed  as  part  of  the  informed  consent  process.  This  should  include 


guidelines  for  troubleshooting  technical  issues,  methods  for  contacting  the  patient  in  the  event  of 
connection  or  equipment  failure  (e.g.,  telephone),  and  contact  information  for  emergency  response 
services  accessible  from  the  patient's  physical  location.5'21  Contingency  plans  are  especially  important  in 
the  case  of  suicide  assessment  delivered  via  VCT,  as  communication  with  clients  may  be  interrupted  and 
the  assessment  may  lead  to  further  action  and  immediate  triage  that  will  involve  third  parties  such  as 
law  enforcement  in  order  to  manage  suicide  behavior  risk  appropriately.21 

When  providing  home-based  TMH  services,  especially  with  patients  who  have  little  experience  with 
technology,  in-home  with  technology  novel  to  the  patient,  a  technical  support  specialist  may  be  useful 
in  delivering  the  equipment,  setting  up  the  technology  correctly,  and  providing  training  on  its  proper 
use.  Family  members  or  close  friends  may  be  useful  in  assisting  patients  with  operating  the  VCT 
equipment,  and  can  also  be  instrumental  as  a  local  collaborator  for  emergencies  to  help  with  risk 
management.5,55  Furthermore,  third  parties  can  be  useful  in  the  assessment  process  by  providing 
additional  information  about  the  patient,  offering  a  unique  perspective  on  client  concerns  and 
presenting  symptoms,  and  utilizing  their  knowledge  of  the  client  to  inform  treatment  plans.56  Flowever, 
Luxton  and  colleagues  noted  that  TMFI  assessment  in  a  client's  home  may  not  be  appropriate  for  some 
patients,  especially  when  encountering  the  increased  safety  risks  for  cases  with  a  history  of  self-harm  or 
harm  to  others.5 

Exchange  of  Assessment  Materials  and  Providing  Assessment  Results 

When  providing  VCT-based  assessments,  informed  consent  documents,  privacy  policies,  self-report 
questionnaires,  assessment  related  task  forms,  and  other  materials,  such  as  assessment  results,  need  to 
be  exchanged  between  clients  and  practitioners  in  a  manner  that  is  efficient  and  secure.  In  some  cases, 
these  types  of  documents  may  need  to  be  mailed,  faxed,  or  checked-out  from  a  satellite  location.  The 
exchange  of  assessment-associated  documents  can  also  be  conducted  electronically.  For  example,  email 
can  be  used  to  deliver  documents  remotely  to  a  tablet  computer  provided  to  the  patient  for  a  digital 
signature  and  captured  as  an  electronic  record.  The  informed  consent  process  can  also  be  completed 
using  websites  and  digital  signature  technology,  for  example  through  the  comprehensive  electronic 
informed  consent  and  participant  management  database  Research  Permission  Management  System.57 
Flarmell  and  colleagues  have  developed  a  web-aided  multimedia  consent  procedure  that  may  be  more 
effective  than  traditional  forms  of  consent  among  a  sample  of  individuals  with  schizophrenia.58  This  web 
format  did  not  affect  comprehension  among  non-psychotic  individuals,  which  was  excellent  regardless 
of  format,  suggesting  that  web-aided,  multimedia  informed  consent  may  be  most  appropriate  for  those 
with  comprehension  impairments. 

If  a  third  party  is  involved  during  the  assessment  process,  such  as  a  family  member,  a  signed  release  of 
information  form  may  be  needed.  Releases  can  be  faxed  or  mailed  to  clients  in  a  self-addressed  and 
stamped  business  envelope  or  signed  at  an  initial  in-person  meeting,  and  practitioners  must  plan  for 
these  procedures  and  discuss  them  with  clients  as  part  of  an  ongoing  informed  consent  process. 
Regardless  of  how  assessment  materials  are  to  be  exchanged,  patients  should  be  made  aware  of  the 
risks  that  this  may  pose  for  their  confidentiality  based  on  the  mechanisms  used. 


As  with  conventional  in-person  assessment,  assessment  results  can  be  provided  verbally  over  VCT. 
Written  feedback  can  also  be  provided  using  technologies,  such  as  VCT  software  that  provides  display  of 
digital  documents,  or  in  conjunction  with  other  internet  technologies  (e.g.,  opening  the  report  on  the 
practitioner's  computer  and  using  a  "screen  share"  option,  encrypted  e-mail  or  document  transfer, 
logging  into  a  secure  web-portal  on  a  secondary  computer,  etc.). 

Practitioner's  must  utilize  clinical  judgment  and  incorporate  available  scientific  knowledge  regarding  the 
appropriateness  of  providing  assessment  feedback  via  VCT,28  a  process  which  may  vary  significantly  for 
different  tests,  patients,  and  practice  guidelines.  Interpretation  and  explanation  of  results  in  TMH 
contexts  must  incorporate  such  factors  as  possible  adaptations  to  administration  protocols,  interference 
from  technology  and  hardware  functioning,  and  distracters  in  the  patient's  physical  environment.28  Just 
as  practitioners  can  note  in  written  reports  the  potential  limitations  of  adapting  tests  for  TMH  that  were 
originally  developed  for  in-person  administration,26  verbal  feedback  provided  via  VCT  should  also  specify 
these  same  limitations  where  appropriate.  Assessment  measures  may  also  have  specific  instructions  for 
providing  feedback  based  on  results  in  their  technical  manuals,  and  different  jurisdictions  may  establish 
minimum  requirements  for  feedback  of  assessment  results,  similar  to  the  requirement  for  in-person 
delivery  of  HIV  testing  results  in  the  field  of  preventive  medicine.  Providing  feedback  via  VCT  may  not  be 
appropriate  for  every  client,  and  practitioners  should  utilize  clinical  judgment  carefully  in  order  to  assure 
the  safety  of  patients  and  collateral  people  who  may  be  in  the  patient's  physical  environment.  For 
example,  providing  verbal  feedback  via  VCT  for  a  client  with  a  complex  personality  disorder  may 
significantly  increase  the  likelihood  of  self-harm  or  aggressive  behavior.  In  conclusion,  clinical  decisions 
about  providing  assessment  feedback  via  VCT  should  be  integrated  into  the  larger  decision-making 
process  about  the  appropriateness  of  remote  assessment  and  the  unique  factors  relevant  to  this  aspect 
of  TMH  practice. 

Conclusions 

Clinical  assessment  via  VCT  or  other  telehealth  technologies  provides  many  benefits  to  both  patients 
and  clinicians.  VCT-based  assessments  can  help  overcome  barriers  to  care  by  providing  access  to  clinical 
services  in  rural  or  underserved  areas,  increasing  the  availability  of  practitioners  familiar  with  the 
nuances  of  a  particular  culture  or  population,  and  allowing  access  to  providers  who  specialize  in 
particular  areas  of  assessment.  Clinical  assessment  via  VCT  can  also  be  a  solution  to  language  barriers  by 
connecting  patients  with  practitioners  who  are  familiar  with  the  patient's  language  or  by  connecting 
hearing  impaired  patients  to  clinicians  who  are  capable  of  conducting  clinical  assessment  via  American 
Sign  Language.  Clinical  assessment  via  VCT  also  allows  clinicians  to  expand  their  practice  and  provide 
services  that  they  otherwise  would  be  limited  by  due  to  their  geographical  location. 

Further  technological  advances  will  continue  to  expand  the  capabilities  of  clinical  assessment  via  VCT.  As 
stated  earlier  in  this  chapter,  the  video  camera  capabilities  on  modern  mobile  smart  devices 
(smartphones  and  tablet  devices)  have  brought  a  whole  new  way  to  provide  VCT  services.  Further 
adoption  and  integration  of  assessments  delivered  via  mobile  devices  can  also  be  expected  in  the  years 
ahead.59  The  advantages  of  these  tools  are  that  they  can  augment  traditional  VCT  clinical  assessments 


by  providing  continuous  sensing  and  unobtrusive  monitoring  of  behavior  and  symptoms  that  can  be 
tracked  over  time.  In  some  examples,  the  devices  can  provide  real-time  feedback  to  users  or  results 
from  assessments  can  be  uploaded  to  practitioners  for  review  and  monitoring.  Further,  many  modern 
mobile  devices  have  the  capability  to  connect  to  external  hardware  devices,  such  as  biofeedback 
sensors,  for  monitoring  physiological  signals.  Smartphone  apps  can  also  be  programmed  to  respond  to 
critical  items  in  self-assessments  to  auto-detect  significant  distress  and,  when  appropriate,  offer  one- 
touch  contact  to  a  support  hotline.59  Moreover,  other  emerging  technologies,  such  as  the  use  of  virtual 
reality  environments  and  the  application  of  artificial  intelligence  to  assessment  systems  will  further 
expand  capabilities.60 

The  use  of  VCT  for  conducting  clinical  assessment  can  be  expected  to  become  more  common  in  parallel 
with  the  overall  growth  and  expansion  of  TMH  practice.  The  increasing  availability  of  telehealth 
technologies  and  the  budding  evidence-base  in  support  of  their  use  for  remote  clinical  assessment  will 
continue  to  advance  capabilities.  As  outlined  in  this  chapter,  there  are  many  factors  that  clinicians  and 
clinician  managers  need  to  consider  in  order  to  assure  optimal  VCT-based  clinical  assessments.  All  of 
these  issues  can  be  effectively  addressed  given  attention  to  available  best-practices  and  appropriate 
training. 
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Home-based  telemental  health  (TMH)  treatments  have  the  potential  to  address  current  and 
future  health  needs  of  military  service  members,  veterans,  and  their  families,  especially  for 
those  who  live  in  rural  or  underserved  areas.  The  use  of  home-based  TMH  treatments  to 
address  the  behavioral  health  care  needs  of  U.S.  military  healthcare  beneficiaries  is  not 
presently  considered  standard  of  care  in  the  Military  Health  System.  The  feasibility,  safety,  and 
clinical  efficacy  of  home-based  TMH  treatments  must  be  established  before  broad  dissemina¬ 
tion  of  home-based  treatment  programs  can  be  implemented.  This  paper  describes  the  design, 
methodology,  and  protocol  of  a  clinical  trial  that  compares  in-office  to  home-based  Behavioral 
Activation  for  Depression  (BATD)  treatment  delivered  via  web-based  video  technology  for 
service  members  and  veterans  with  depression.  This  grant  funded  three-year  randomized 
clinical  trial  is  being  conducted  at  the  National  Center  for  Telehealth  and  Technology  at 
Joint-base  Lewis-McChord  and  at  the  Portland  VA  Medical  Center.  Best  practice  recommen¬ 
dations  regarding  the  implementation  of  in-home  telehealth  in  the  military  setting  as  well  as 
the  cultural  and  contextual  factors  of  providing  in-home  care  to  active  duty  and  veteran 
military  populations  are  also  discussed. 

Published  by  Elsevier  Inc. 


1.  Introduction 

There  is  mounting  evidence  supporting  the  clinical  effec¬ 
tiveness  of  telemental  health  (TMH)  treatments  [31,33]  as  well 
as  patient  and  provider  satisfaction  with  TMH  [5,35],  The 
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evidence  base  supporting  home-based  telemental  health 
(HBTMH)  is  also  growing,  and  HBTMH  services  are  expanding 
across  diverse  care  settings  including  the  VA  Health  Care 
System  [  11  j.  HBTMH  treatment  options  have  multiple  benefits: 
they  can  improve  access  to  care  services,  reduce  the  burden  of 
travel  expenses,  eliminate  wait  times,  and  reduce  time  away 
from  work  to  attend  appointments.  Stigma  associated  with 
mental  health  conditions  is  another  barrier  to  care  that  may 
influence  willingness  to  seek  mental  health  treatment  The 
option  to  receive  care  in  the  comfort  and  privacy  of  the  home  is 
one  way  to  combat  this  problem  [40]. 

Mental  health  treatments  provided  directly  to  the  homes 
of  U.S.  military  personnel  are  not  presently  the  standard  of 
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care  in  the  Military  Health  System  (MHS).  Clinical  research  is 
needed  to  test  the  feasibility,  safety,  and  effectiveness  of 
HBTMH  treatments  in  the  military  setting  in  order  to  inform 
policies  regarding  the  adoption  and  expansion  of  HBTMH.  To 
address  this  need,  we  are  conducting  a  randomized  clinical 
trial  (RCT)  that  compares  Behavioral  Activation  Treatment 
for  Depression  (BATD;  [14])  delivered  in-office  to  BATD 
delivered  via  webcams  to  the  homes  of  U.S.  military  service 
members  and  veterans  with  depression. 

Behavioral  Activation  for  depression  was  selected  as  the 
treatment  in  our  trial  for  several  reasons.  First,  military 
personnel  may  be  highly  agreeable  to  BA  as  a  treatment  option. 
Behavioral  Activation  is  based  on  a  behavioral  conceptualization 
of  depression  which  posits  that  depression  is  an  understandable 
response  to  negative  life  events  and  difficult  environments  [13]. 
This  stance,  that  “depression  makes  sense,”  renders  BA  less 
stigmatizing  than  other  treatments  because  it  does  not  assume 
weakness  or  disorder  on  the  part  of  the  patient  [37],  Behavioral 
Activation  is  also  an  action-oriented  treatment  that  that  may  be 
particularly  acceptable  to  physically  active  military  service 
members.  Second,  BA  has  considerable  empirical  support 
for  the  treatment  of  depression  among  both  civilians  (see 
[13])  and  Veteran  populations  [7]  as  well  preliminary  support 
as  a  treatment  for  PTSD  [12,20].  Third,  depression  is  a  highly 
prevalent  mental  health  condition  in  both  the  military  and 
Veterans  populations  and  it  is  the  most  frequent  reason  for 
psychiatric  hospitalization  in  both  the  active  and  reserve 
components  of  the  U.S.  Armed  Forces  [27], 

With  this  paper  we  describe  the  design,  methodology, 
safety  management,  and  treatment  protocols  for  an  in-progress 
Military  Operational  Medicine  and  Research  Programs  (MOMRP) 
grant  funded  multi-site  clinical  trial  of  HBTMH  treatment.  The 
trial  is  registered  on  the  United  States  National  Institutes  of 
Health  Clinical  Trials  Registry,  (ClinicalTrials.gov  Identifier 
#NCT01 599585)  available  online  at:  http://clinicaltrials. 
gov/show/NCTOl  599585.  In  addition  to  testing  the  effec¬ 
tiveness  of  a  HBTMH  treatment,  the  study  provides  data  on 
patient  satisfaction  with  a  home-based  care  and  it  advances 
the  knowledge  base  regarding  the  safety  and  risk  manage¬ 
ment  procedures  of  home-based  treatments  in  both  the 
military  and  VA  settings.  The  study  also  tests  the  feasibility 
of  readily  available  synchronous  videoconferencing  technolo¬ 
gies  (i.e.,  webcams  and  laptop  computers)  to  provide  care  to 
military  personnel  and  veterans  in  the  home.  This  paper  should 
be  particularly  useful  to  researchers  who  are  interested  in  the 
technical  aspects  of  implementing  clinical  telehealth  research 
in  the  military,  VA,  and  other  health  care  settings.  The  paper 
also  highlights  the  procedures  for  applying  a  two-group 
non-inferiority  trial  design  to  establish  a  novel  or  alternative 
mode  of  treatment  delivery  as  standard  of  care. 

2.  Research  design  and  methods 

2.J.  Study  design 

This  RCT  is  a  two-group  non-inferiority  design  that  com¬ 
pares  the  effectiveness  of  BATD  delivered  via  web-cam  to 
standard  in-office  BATD.  The  study  is  being  conducted  at  the 
National  Center  for  Telehealth  and  Technology  (T2)  located 
at  Joint  Base  Lewis-McChord  (JBLM;  Fort  Lewis,  WA)  and  at 
the  Portland  Veterans  Administration  Medical  Center  (PVA; 


Portland,  OR).  We  followed  the  Consolidated  Standards  of 
Reporting  Trials  (CONSORT)  guidelines  in  developing  the 
protocol,  and  our  procedures  adhere  to  the  principles  and 
recommendations  of  the  World  Medical  Association,  Decla¬ 
ration  of  Helsinki  Ethical  Principles  for  Medical  Research 
Involving  Human  Subjects,  as  well  as  all  applicable  Codes  of 
Federal  Regulation  and  Department  of  Army  Regulations. 
The  research  protocols  were  approved  by  the  Institutional 
Review  Board  (IRB)  at  each  site  and  the  protocols  underwent 
separate  review  processes  by  the  Army  Human  Research 
Protection  Office  (HRPO). 

The  study’s  conceptual  design  is  shown  in  Fig.  1.  Eligible 
patients  have  an  equal  chance  of  being  randomized  to  either 
the  in-office  or  in-home  treatment  groups.  All  patients  are 
provided  with  8  sessions  of  BATD  that  is  guided  by  a  treatment 
protocol  manual.  Patients  in  both  intervention  groups  follow 
the  same  assessment  schedule  with  assessments  at  baseline, 
mid-treatment,  1  week  post-treatment,  and  3  months  post 
treatment  (see  Table  1).  Treatment  clinicians  are  blinded  to 
clinical  assessment  results.  This  design  characteristic,  along 
with  the  treatment  fidelity  process  (described  in  Section  2.5), 
was  implemented  to  prevent  the  study  clinicians  from  system¬ 
atically  altering  treatment  delivery  due  to  potential  biases  in 
favor  of  or  against  either  diagnostic  group. 

2.2.  Setting 

Joint  Base  Lewis-McChord  (JBLM)  is  a  large  U.S.  Army 
and  U.S.  Air  Force  base  that  is  home  to  Madigan  Army 
Medical  Center  (MAMC),  a  Regional  Medical  Center  and 
teaching  hospital  that  serves  more  than  108,000  beneficia¬ 
ries  across  a  network  of  military  treatment  facilities  located 
throughout  Washington  State,  Oregon,  and  California.  The 


Fig.  1.  Study  flow  chart. 
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Table  1 

Schedule  of  measures  and  survey  instruments. 


Baseline 

Midpoint 

Post-Tx 

3-month  follow-up 

Measure 

Initial 

Wl<  1 

Wk  2 

Wk  3 

Wk  4 

Wk  5 

Wk  6 

Wk  7 

Wk  8 

Final 

SCID-I/P 

X 

X 

X 

X 

BDI-II 

X 

X 

X 

X 

BHS 

X 

X 

X 

X 

BAI 

X 

X 

X 

X 

LS 

X 

X 

X 

X 

PCL-M 

X 

X 

X 

X 

IASMHS 

X 

X 

X 

Safety 

Measures 

X 

X 

X 

X 

X 

X 

X 

X 

X 

CSQ 

Technology 

Questionnaire 

X 

X 

X 

Note.  SC1D-I/P  =  Structured  Clinical  Interview  for  the  DSM-IV  Axis  I  Disorders,  Research  Version,  Patient  Edition;  BHS  =  Beck  Hopelessness  Scale;  BD1-II  =  Beck 
Depression  Inventory-11;  BA1  =  Beck  Anxiety  Inventory;  LS  =  Loneliness  Scale;  PCL-M  =  PTSD  Checklist  —  Military;  IASMHS  =  Inventory  of  Attitudes  Toward 
Seeking  Mental  Health  Services;  CSQ  =  Client  Satisfaction  Questionnaire;  Technology  Questionnaire  =  Computer  and  Audiovisual  Technology  Questionnaire. 


National  Center  for  Telehealth  &  Technology  is  part  of  the 
Defense  Centers  of  Excellence  for  Psychological  Health  & 
Traumatic  Brain  Injury  and  the  Military  Health  System  (MHS). 
It  is  co-located  with  MAMC  on  JBLM.  The  National  Center  for 
Telehealth  &  Technology's  mission  is  to  lead  in  the  development 
and  research  of  telehealth  and  health  technology  solutions  for 
the  military  community.  Study  patients  at  T2  are  comprised  of 
active-duty,  reserve,  and  National  Guard  service  members  who 
are  eligible  to  receive  health  care  through  the  MHS.  The  study 
patients  at  the  PVA  site  are  military  veterans  receiving  health 
care  services  through  the  VA  hospital  in  central  Portland 
(Veterans  Integrated  Service  Network  [VISN]  20).  These  vet¬ 
erans  reside  throughout  various  towns  and  cities  in  Northwest 
Oregon  and  Southwest  Washington  State.  The  study  teams  at 
each  site  meet  every  two  weeks  via  videoconferencing  to  assure 
parallel  operations  and  assess  study  progress. 


2.3.  Participants  and  enrollment  methods 

Approximately  120  ( n  =  90  at  JBLM;  n  =  30  at  PVA) 
patients  will  be  recruited  with  an  anticipated  treatment 
completion  rate  of  108  (90%;  54  per  treatment  group). 
Patients  are  male  and  female  members  of  the  U.S.  Armed 
Forces  recruited  from  MAMC  and  the  larger  JBLM  community 
as  well  as  U.S.  military  veterans  recruited  at  the  PVA  site. 
Study  eligibility  depends  in  part  on  whether  the  patient  has 
high  speed  internet  access  at  home  (384kbs  or  greater)  as 
well  as  a  private  space  for  treatment  sessions  (complete 
inclusion  and  exclusion  criteria  can  be  found  in  Table  2). 
Patients  that  are  randomized  to  the  in-office  treatment  group 
are  seen  in  a  traditional  face-to-face  clinical  office  setting  at 
T2  or  PVA.  Patients  assigned  to  the  in-home  treatment  group 
are  issued  a  Dell  Precision  M6500  laptop  computer,  Tandberg 
Precision  High  Definition  webcam,  and  auxiliary  equipment 
(e.g.  mouse,  charging  station  and  power  cables)  that  they 
connect  to  their  own  private  internet  access  (either  wireless 
or  wired  connection).  The  lap-tops  are  password  protected 
and  functionality  is  restricted  so  that  unauthorized  software 
cannot  be  loaded  onto  them.  The  videoconferencing  software 
being  used  is  Cisco  Jabber  Video  for  Telepresence.  This  software 
has  embedded  encryption  features  that  meet  Health  Insurance 


Portability  and  Accountability  Act  requirements  and  it  is 
authorized  for  use  by  the  U.S.  Army. 

The  primary  referral  sources  for  study  patients  are 
clinical  providers  within  behavioral  health,  primary  care, 
and  operational  medicine  service  programs  at  JBLM  and 
PVA  (e.g.,  psychologists,  psychiatrists,  physicians,  social 
workers,  nurse  practitioners,  and  nurses).  Military  chaplains 
on  JBLM  also  serve  as  a  recruitment  source.  These  referring 
professionals  are  not  affiliated  with  the  trial,  but  have  been 
informed  about  referral  procedures  during  informational 
presentations  by  study  staff.  Additional  recruitment  strategies 
include  flyers  and  banners  as  well  as  the  use  of  social  media 
campaigns  (i.e.  Facebook,  Twitter,  and  Linkedln)  that  target 
treatment  providers  who  could  make  patient  referrals.  Patient 
recruitment  began  in  August  of  201 2. 

Following  referral,  the  study  coordinator  conducts  a  brief 
phone  screen  and  schedules  each  potential  patient  for  an 
individual  meeting  with  an  outcomes  assessor  to  complete  the 
informed  consent  process  and  discuss  study  procedures  in 
detail.  Participation  is  discussed  as  entirely  voluntary  without 
negative  consequences  for  withdrawal.  At  the  PVA  site, 
patients  receive  $20  for  each  of  the  first  three  assessment 
visits,  and  $40  for  completion  of  the  3  month  follow-up 
assessment.  Patients  at  the  JBLM  site  cannot  be  compensated 
for  their  time  per  US  Army  regulations.  Each  patient's  capacity 
to  consent  and  answer  any  questions  about  study  procedures  is 
monitored  during  the  course  of  treatment  and  during  assess¬ 
ment  visits  as  well. 

After  patients  complete  the  baseline  assessment  (see 
Section  2.7),  those  meeting  eligibility  criteria  are  assigned  to 
treatment  condition  by  the  study  coordinator  (who  is  not 
blinded  to  condition)  according  to  the  pre-determined  ran¬ 
domization  schedule.  The  treatment  condition  is  determined 
for  both  sites  by  using  a  computer  generated  table  with  a  block 
(n  =  10)  randomization  algorithm.  This  procedure  allows 
each  patient  to  have  equal  chance  of  being  assigned  to  either 
of  the  two  groups  while  assuring  equal  distribution  of  patients 
to  the  two  conditions  over  the  course  of  the  study.  We  did 
not  use  a  stratification  scheme  based  on  site,  demographic 
variables,  or  medication  status.  We  anticipate  that  demograph¬ 
ic  characteristics,  such  as  gender,  race,  and  age  will  be  reflective 
of  the  military  and  veteran  population  from  which  we  are 
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Table  2 

Inclusion  and  exclusion  criteria  for  patient  enrollment. 

Inclusion  criteria 

(a)  Met  diagnostic  criteria  for  minor  depressive  disorder  or  major 
depressive  disorder,  as  determined  by  the  SCID-I/P 

(b)  High  speed  internet  access  at  home  (384  kbs  minimum) 

(c)  If  taking  psychoactive  medications,  has  maintained  a  stable 
regimen  for  a  minimum  of  30  days  prior  to  study  entry 

(d)  Informed  consent  read  and  signed 
Exclusion  criteria 

(a)  Currently  undergoing  psychotherapy  for  depression 

(b)  <18  or  >65  years  of  age 

(c)  Active  psychotic  symptoms/disorder  as  determined  by  the  SCID-I/P 

(d)  Dysthymic  disorder  as  determined  by  the  SCID-I/P 

(e)  Current  suicidal  ideation  with  intent  or  recent  (within  six  months) 
history  of  a  suicide  attempt 

(f)  History  of  organic  mental  disorder 

(g)  Current  substance  dependence  as  determined  by  the  SCID-I/P 
(lifetime  substance  dependence  or  substance  abuse  will  not  be 
excluded) 

(h)  History  of  violence  or  poor  impulse  control 

(i)  Significant  ongoing  stressors  that  require  urgent  crisis  intervention 

(j)  Have  a  living  arrangement  that  will  not  permit  the  use  of  a  private 
space  to  participate  in  the  study 


sampling.  These  variables  will  be  treated  as  covariates  in  our 
analyses  (See  Section  2.10).  We  will  examine  potential 
differences  in  medication  use  between  groups  in  post-hoc 
analyses. 

2.4.  Clinician  preparation  and  training 

Assessments  are  conducted  by  outcomes  assessors  that 
have  been  blinded  to  treatment  condition.  Assessors  are 
eligible  to  meet  with  patients  after  completing  specific 
assessment  training  protocols  (i.e„  literature  review,  assess¬ 
ment  training  videos,  DSM-1V  TR  review,  taped  practice 
sessions  with  expert  review,  and  role  plays).  Study  clinicians 
are  credentialed  healthcare  providers  at  MAMC  or  PVA.  All 
study  clinicians  are  doctoral  level  clinical  psychologists  who 
have  completed  specific  training  requirements  for  both 
BATD  as  well  as  use  of  the  TMH  technology  and  equipment. 
BATD  training  consists  of  an  extensive  literature  review  (both 
theoretical  and  empirical),  completion  of  mock  sessions,  and 
attendance  at  a  two  day  intensive  training  workshop  led  by  Dr. 
Ron  Acierno,  Ph.D.  who  is  one  of  the  authors  of  the  BATD 
protocol.  Training  in  TMH  technology  and  equipment  consists 
of  test  calls,  troubleshooting  practice,  and  equipment  manual 
review.  All  training  requirements  are  completed  before  a 
provider  is  allowed  to  actively  treat  study  patients. 

2.5.  Behavioral  Activation  treatment  protocol 

BATD  and  other  Behavioral  Activation  protocols  originated 
from  behavior  analytic  models  of  classical  and  operant  condi¬ 
tioning  [25,30,36]  and  the  behavioral  component  of  cognitive 
therapy  for  depression  [1,15-17],  Behavior  analytic  theoiy 
posits  that  depression  develops  when  learned  behavioral 
contingencies  fail  to  produce  stable,  diverse,  and  reinforcing 
environmental  consequences  [13],  This  can  occur  in  a  wide 
range  of  contexts  (e.g.,  trauma,  loss,  daily  stressors)  and  is 
likely  modulated  by  biological  predispositions.  When  an 
individual's  behavior  no  longer  produces  reinforcing  conse¬ 
quences,  a  reduction  in  the  frequency  of  the  target  behavior 


occurs.  Often,  this  can  occur  in  parallel  with  an  increase  in 
the  frequency  of  other  maladaptive  behaviors  associated 
with  that  response,  including  withdrawal,  negative  internal 
affective  experiences,  and  ultimately,  symptoms  of  depres¬ 
sion.  BATD  aims  to  reengage  depressed  individuals  in  their 
lives  through  focused,  values-based  activation  strategies. 
These  strategies  counter  patterns  of  negative  affect,  with¬ 
drawal,  and  inactivity  by  reestablishing  contact  with  natu- 
ralistically  reinforcing  consequences  for  adaptive  behavior 
that  alleviates  depressed  mood  and  creates  stable  patterns 
for  accessing  reinforcing  consequences. 

The  BATD  treatment  protocol  used  in  the  present  trial  is 
based  on  a  revised  BATD  treatment  manual  by  Lejuez  et  al. 
[14],  The  protocol  prescribes  8-sessions  of  BATD  that  can  be 
delivered  either  in-person  or  by  VCT.  In  the  first  session, 
patients  are  provided  with  psychoeducation  about  depres¬ 
sion  and  introduced  to  the  treatment  rationale  and  the  role 
and  importance  of  daily  monitoring  for  the  duration  of  the 
treatment.  In  the  second  session,  previous  content  is  reviewed, 
followed  by  introductions  to  the  concepts  of  values  and  activity 
planning.  With  regards  to  values,  clinicians  utilize  a  series  of 
prompts  and  writing  tasks  to  encourage  patients  to  identify 
their  personal  values  within  five  different  major  life  domains 
(i.e.,  relationships,  education/career,  recreation/interests, 
mind/body/spirituality,  and  daily  responsibilities).  Values 
are  defined  as  ongoing,  meaningful  patterns  of  action  and 
are  contrasted  with  goals,  which  have  an  endpoint.  Patients 
then  collaborate  with  clinicians  to  devise  lists  of  activities 
that  exemplify  their  values.  For  example,  if  a  patient  values 
spending  quality  time  with  his  children,  specific  activities 
might  include  taking  them  to  the  park  for  a  game  of  ‘catch,’ 
reading  3  short  bedtime  stories  to  them  each  night,  and 
spending  half  of  a  hour  helping  them  to  complete  homework 
at  approximately  6:00  pm  each  day.  Activity  planning  is  the 
process  of  collaboratively  scheduling  these  activities  in  ad¬ 
vance  in  order  to  maximize  the  potential  for  contact  with 
naturally  occurring  reinforcement  in  a  patient's  day-to-day 
life.  In  sessions  three  through  eight,  the  treatment  rationale 
is  continuously  reviewed,  and  patients  are  asked  to  schedule 
more  and  varied  values-consistent  activities  using  a  daily 
planner  of  their  choice  (planners  are  provided  for  use  with 
treatment,  but  patients  are  encouraged  to  utilize  established 
planners  or  scheduling  systems  [e.g.,  their  smartphone]  to 
increase  the  chances  of  regular  use).  Final  sessions  are  also 
used  to  address  issues  related  to  termination,  treatment 
progress,  and  ways  to  use  what  has  been  learned  in  treatment 
for  relapse  prevention. 

The  present  study's  treatment  protocol  also  contains  specific 
provisions  for  VCT-based  treatment  delivery  such  as  equipment 
set-up,  procedures  for  initiating  the  VCT  sessions,  and  steps  to 
take  in  the  event  of  disrupted  service.  All  clinical  procedures  for 
the  in-office  and  in-home  conditions  are  identical. 

2.6.  Treatment  fidelity 

To  assure  adherence  to  the  treatment  protocol,  treatment 
providers  complete  session-by-session  “Adherence  Checklists" 
that  highlight  the  key  elements  of  each  session  as  well  as 
homework  that  is  assigned.  Treatment  providers  also  partici¬ 
pate  in  weekly  individual  and  group  supervision  and  they 
attend  weekly  cross-site  (i.e.,  JBLM  and  PVA)  case  consultation 
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meetings.  To  assess  adherence  to  the  treatment  protocol,  all 
treatment  sessions  are  digitally  recorded.  Sessions  at  MAMC 
are  video  recorded  onto  DVD  (although  the  video  captures  the 
clinician  only  along  with  audio  for  both  clinician  and  patient), 
whereas  sessions  at  PVA  are  audio  recorded  only  (per  VA 
policy).  Ten  percent  of  these  session  recordings  are  randomly 
selected  and  sent  to  an  expert  fidelity  reviewer  on  a  monthly 
basis  (Dr.  Ron  Acierno).  The  fidelity  reviewer  was  selected 
for  his  expertise  in  the  delivery  of  BATD  to  service  members 
and  veterans  via  telehealth.  The  reviewer  codes  each  session 
recording  for  compliance  based  on  a  treatment  fidelity  checklist 
delineating  the  essential  therapeutic  components  that  must  be 
delivered  in  each  session  of  BATD. 

Description  of  the  recordings  and  fidelity  review  is  provided 
in  the  informed  consent  process  at  both  study  sites.  VA  study 
clinicians  are  also  required  to  obtain  patient  consent  for 
recording  using  the  VA  Form  10-3203,  “Consent  for  use  of 
picture  and/or  voice”  in  addition  to  their  IRB  approved 
Informed  Consent  Form.  At  JBLM,  the  digital  recordings  are 
retained  for  5  years  after  the  publication  of  results.  At  PVAMC, 
in  accordance  with  VA  policy,  digital  recordings  are  retained 
indefinitely. 

2.7.  Safety  management  protocol 

During  the  baseline  assessment,  outcomes  assessors 
conduct  a  thorough  suicide  risk  assessment  to  determine 
level  of  risk  per  MAMC  standard  operating  procedure  (see 
[20]  for  further  description).  Level  of  risk  depends  on  a 
combination  of  risk  correlates  (e.g.,  substance  abuse,  signif¬ 
icant  psychosocial  stressors);  factors  related  to  suicide  desire 
and  ideation  (e.g.,  articulated  reasons  for  living,  passive 
thoughts  of  attempt);  and  resolved  plans  and  preparation 
(e.g.,  available  means,  specific  plans).  Patients  are  asked  to 
identify  a  third  party  (e.g.,  family  member  or  friend)  who 
may  be  able  to  assist  in  cases  of  emergency  or  imminent  risk. 
At  JBLM,  service  members  are  also  asked  to  provide  the 
contact  information  for  their  immediate  commanding  offi¬ 
cers  in  case  of  emergency  or  elevated  risk  necessitating 
command  notification,  per  Army  regulation.  This  additional 
exception  to  confidentiality  in  the  military  setting  is  thor¬ 
oughly  reviewed  as  part  of  informed  consent.  Lastly,  for 
patients  randomized  to  the  in-home  condition,  clinicians 
identify  the  best  contact  information  for  law  enforcement 
and  emergency  services  nearest  each  patient's  home  address 
for  use  in  case  of  emergency. 

Suicidal  risk  is  re-assessed  during  the  first  treatment  session 
(using  the  same  standard  operating  procedure  described 
above).  The  assessment  is  also  conducted  during  subsequent 
assessment  and/or  treatment  sessions  for  patients  identified  to 
be  at  intermediate  or  high  risk  or  if  a  patient  indicates  a  change 
in  the  severity  or  frequency  of  suicidal  ideation.  Individuals 
advance  from  low  to  higher  levels  of  risk  based  on  frequency, 
intensity,  and  duration  of  ideation,  as  well  as  presence  of 
intent  and/or  plan  or  evaluation  of  risk  factors.  For  all  patients, 
regardless  of  their  initial  risk  level  determination,  ideation  and 
other  signs  of  risk  for  self-harm  are  also  monitored  at  each 
session  by  means  of  a  treatment  session  checklist  developed 
specifically  for  this  study  (see  Attachment  1 ).  Relevant  ques¬ 
tions  assess  correlates  of  safety  risk,  alcohol  or  substance  use, 
appearance  of  being  disoriented  or  upset;  reports  of  suicidal 


desire  and  ideation,  and  whether  a  weapon  was  observed. 
This  checklist  also  includes  questions  that  assess  aspects  of 
patient  safety  other  than  suicide  risk.  For  example,  some 
questions  assess  more  general  safety-related  questions,  such 
as,  “Is  anyone  else  at  home  today?",  whereas  some  questions 
are  more  specific,  such  as  “Did  the  patient  indicate  intent  to 
harm  others?”  Finally,  in  line  with  Luxton  and  colleagues' 
[18]  definition  of  safety,  this  checklist  also  addresses  matters 
of  privacy  and  confidentiality  (e.g.,  “Do  you  feel  that  your 
environment  is  safe  and  private?"),  as  well  as  technology, 
equipment  and  connectivity  (e.g.,  “What  number  can  1  reach  you 
at  if  we  get  disconnected?",  “Were  there  problems  initiating/ 
maintaining  the  webcam  connection?").  The  checklist  also 
contains  space  for  clinician  comments. 

If  any  study  clinician  or  assessor  becomes  aware  of  any 
elevation  in  patient  risk  for  suicide  or  violent  behavior, 
established  written  safety  protocols  are  followed  according  to 
the  regulations  of  the  site  responsible  for  that  patient.  This 
may  include  developing  a  detailed  safety  plan  with  patients, 
modifying  risk  factors  and  removing  lethal  means  for  suicide 
or  violent  behavior,  involving  the  third  party  identified  by 
patients  to  help  with  enacting  the  safety  plan,  modifying  risk 
factors,  assisting  with  patient  safety  until  the  patient  is 
transferred  to  emergency  services,  involving  the  patient's 
commander  and  assigned  unit,  and  transferring  the  patient  to 
inpatient  care.  At  the  baseline  assessment,  high  short  term  risk 
for  suicide  behavior  would  preclude  participation  in  the  study 
in  favor  of  more  acute,  crisis-focused  care.  Elevated  risk  during 
treatment  is  managed  via  consultation  by  the  treatment  team, 
who  collectively  determine  whether  the  patient  in  question 
can  be  safely  and  effectively  treated  within  the  confines  of  the 
treatment  protocol  or  whether  more  acute  and/or  intensive 
care  is  warranted.  For  participants  who  are  determined  to  be  in 
the  “high  risk”  category  on  the  suicide  assessment  but  are  not 
presently  experiencing  a  crisis,  the  treatment  provider  works 
to  establish  a  safety  plan  with  the  participant  while  coordinat¬ 
ing  the  transfer  of  the  participant  out  of  the  study  and  to 
appropriate  care  sen/ices.  Additional  information  regarding 
suicide  risk  management  and  assessment  procedures  used 
in  the  present  trial  are  described  by  Luxton,  O'Brien,  Pruitt, 
Johnson,  &  Kramer  (In  Press). 

All  assessors  and  providers  adhere  to  Federal  Health 
Insurance  Portability  and  Accountability  Act  regulations  as 
well  as  their  state  (Washington/Oregon)  and  military/VA 
requirements  of  confidentiality,  including  exceptions  and 
reporting  of  imminent  risk  of  harm  to  self  or  others,  including 
harm  to  vulnerable  populations.  The  staff  at  T2  also  complies 
with  Army  Medical  Command  (MEDCOM)  regulations  man¬ 
dating  that  providers  in  the  Military  Health  System  must  follow 
additional  mandatoiy  reporting  requirements  pertaining  to 
substance  use,  sexual  assault,  and  domestic  violence  which 
may  necessitate  notification  of  an  individual's  unit  commander. 

2.8.  Assessments  and  measures 

After  obtaining  informed  consent,  the  condition  naive 
outcomes  assessor  determines  each  person's  study  eligibility  by 
asking  a  series  of  questions  related  to  inclusion  and  exclusion 
criteria  and  conducts  an  abbreviated  Structured  Clinical  Inter¬ 
view  for  the  DSM-IV  Axis  I  Disorders,  Research  Version,  Patient 
Edition  (SCID-l/P;  [8])  which  focuses  on  the  mood,  psychotic, 
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and  substance  use  disorder  sections.  If  eligible  for  participation,  a 
demographics  questionnaire  is  administered  along  with  a  set  of 
self-report  questionnaires  which  follow  the  assessment  schedule 
outlined  in  Table  2.  These  self-report  measures  include:  Beck 


Depression  Inventory-II  (BDI-II;  [3]),  Beck  Hopelessness  Scale 
(BHS;  [4]),  Beck  Anxiety  Inventory  (BA1;  [2]),  Loneliness  Scale 
(LS;  [6]),  PTSD  Checklist  -  Military  (PCL-M;  [38]),  Inventory  of 
Attitudes  Toward  Seeking  Mental  Health  Services  (IASMHS; 


Attachment  1 

Home-based  telemental  health  treatment  session  checklist. 

Treatment  Session  Checklist 


Participant  ID: _ Clinician: _ Session  Date: 


Session  #: 


Questions  to  be  asked  at  beginning  of  session  (in-home  telehealth  condition  only) 

1 .  Is  anyone  else  at  home  today  YES  NO  (circle  one) 
a.  If  yes,  who? 


2.  Do  you  anticipate  any  disruptions  during  our  session  today?  YES  NO  {circle  one) 
a.  If  yes,  explain: 


3.  Do  you  feel  that  your  environment  is  safe  and  private?  YES  NO  (circle  one) 
a.  If  yes,  explain: 


4.  What  number  can  I  reach  you  at  if  we  get  disconnected? 


Participant 

1 .  Was  participant  late  to  the  session?  YES  NO  ( circle  one ) 

a.  If  yes,  how  many  minutes  late? _ 

2.  Did  the  participant  cancel  session  early?  YES  NO  {circle  one) 
a.  If  yes,  explain: 


3.  Was  the  session  rescheduled?  YES  NO  {circle  one) 

a.  If  yes,  who  was  it  rescheduled  by?  _ 

4.  Did  the  participant  miss  the  session  without  giving  prior  notice?  YES  NO  {circle  one) 
a.  If  yes,  explain: 


5.  Did  the  participant  appear  disheveled?  N/A  YES  NO  {circle  one) 
1.  If  yes,  explain: 


6.  Did  participant  appear  intoxicated?  N/A  YES  NO  {circle  one) 
1.  If  yes,  explain: 


7.  Did  patient  show  any  signs  of  suicidal  ideation?  YES  NO  {circle  one) 
1.  If  yes,  explain: 


8.  Did  patient  exhibit  self-harm  behavior  during  session?  YES  NO  {circle  one) 
1.  If  yes,  explain: 


9.  Was  there  a  suicide  attempt  since  last  session?  YES  NO  {circle  one) 
1.  If  yes,  explain: 


10.  Did  patient  indicate  intent  to  harm  to  others?  YES  NO  {circle  one) 
1.  If  yes,  explain: 
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1 1 .  Did  participant  become  upset/distressed  during  session?  YES  NO  ( circle  one 

1.  If  yes,  explain: 


Safety  Protocol 

1.  Was  the  safety  protocol  initiated?  YES  NO  (circle  one) 

2.  Was  it  necessary  to  contact  collateral?  YES  NO  ( circle  one) 
a.  If  yes,  check  all  that  apply: 

□  Able  to  contact  the  collateral 

□  Collateral  responded  to  issue 

□  The  collateral  was  helpful 

3.  Was  the  police  non-emergency  line  or  91 1  called?  YES  NO  (circle  one) 
a.  If  yes,  check  all  that  apply: 

□  The  agency  initiated  an  emergency  response 

□  The  agency  chose  not  to  provide  an  emergency  response 

□  The  agency  was  unable  to  provide  an  immediately  emergency  response  (e.g., 
location) 

4.  Was  a  supervisor  notified  or  consulted?  YES  NO  (circle  one) 
a.  If  yes, 

i.  Name  of  supervisor: _ 

ii.  Date  and  time  contacted: 


Environment  (in-home  telehealth  condition  only) 

1.  Were  there  distractions  at  the  patient’s  location  (e.g.,  pets,  children,  cell  phones)? 
YES  NO  (circle  one) 
a.  If  yes,  explain: 


b.  If  yes,  was  this  useful  clinical  information?  YES  NO  (circle  one) 

2.  Was  the  session  interrupted  by  another  person?  YES  NO  (circle  one) 
a.  If  yes,  explain: 


b.  If  yes,  was  this  useful  clinical  information?  YES  NO  (circle  one) 

3.  Any  weapons  observed  during  session?  YES  NO  (circle  one) 
a.  If  yes,  explain: 


b.  If  yes,  was  this  useful  clinical  information?  YES  NO  (circle  one) 

4.  Did  the  participant’s  room  have  adequate  lighting?  YES  NO  (circle  one) 
a.  If  yes,  explain: 


Technology  Issues  (in-home  telehealth  condition  only) 

1.  Were  there  problems  initiating  the  webcam  connection?  YES  NO  (circle  one) 

a.  If  yes,  how  many  minutes  until  connection  made? _ 

b.  Indicate  source  of  problem  (check  all  that  apply) 

_ □  Internet  Connection  (ISP  Problem) _ 


[21]),  Client  Satisfaction  Questionnaire  (CSQ;  [27]),  Computer 
and  Audiovisual  Technology  Questionnaire  (Technology  Ques¬ 
tionnaire;  adapted  by  study  authors  from  [7]),  and  safety 
measures  (recording  any  clinical  safety  concerns  and  adverse 
events  at  each  client  contact).  Additionally,  a  comprehensive 
suicide  risk  assessment  is  conducted  at  the  initial  meeting  by  the 


study  assessor  with  ongoing  monitoring  by  study  therapists  as 
part  of  clinical  risk  management  (see  Section  2.10). 

All  standardized  symptom  inventory  scales  have  been 
previously  used  in  research  with  military  populations  and 
demonstrate  adequate  psychometric  properties  (e.g.,  validity 
and  internal  consistency  reliability  estimates  calculated  in 
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□  Software  problem 

□  Hardware  problem  (PC,  webcam,  microphone) 

□  Unable  to  establish  webcam  connection 

□  Unable  to  follow-up  via  telephone 

□  Unable  to  contact  the  PSP 
Describe: 


2.  Were  there  problems  maintaining  the  webcam  connection?  YES  NO  ( circle  one) 

a.  If  yes,  how  many  minutes  until  connection  made? _ 

b.  Indicate  source  of  problem  (check  all  that  apply) 

□  Internet  Connection  (ISP  Problem) 

□  Software  problem 

□  Hardware  problem  (PC,  webcam,  microphone) 

□  Participant  purposely  terminated  contact 

□  Unable  to  re-establish  webcam  connection 

□  Unable  to  follow-up  via  telephone 

□  Unable  to  contact  the  PSP 
Describe: 


3.  Was  it  necessary  to  contact  participant  by  phone?  YES  NO  (circle  one) 


Other  Information: 


data  collected  from  samples  with  similar  demographic  profiles). 
Research  on  the  Loneliness  Scale  has  demonstrated  sound 
psychometric  outcomes  in  a  large  sample  of  adults  [6],  and 
Mackenzie  et  al.  [21]  presented  initial  validity  evidence  for  the 
IASMHS  with  strong  psychometrics  calculated  from  a  sample  of 
young  adult,  undergraduate  students.  Given  the  frequent  move¬ 
ment  of  military  personnel  between  installations  in  the  United 
States  as  well  as  deployment  to  overseas  locations,  the  study 
procedures  allow  for  the  three  month  follow-up  assessment  to  be 
completed  remotely  if  necessary  via  a  combination  of  telephone 
interview  (SC1D)  and  mail  (self-report  questionnaires). 

The  outcome  assessors  remain  blinded  to  treatment  condi¬ 
tion  throughout  the  course  of  the  study  to  avoid  bias  the  assessor 
may  have  toward  any  one  particular  treatment  condition. 
Procedural  and  physical  barriers  are  used  to  protect  the  assessor 
from  being  inadvertently  exposed  to  information  about  the 
treatment  condition  to  which  patients  have  been  assigned.  For 
example,  treatment  and  assessment  sessions  take  place  in 
different  office  spaces  and  efforts  are  taken  to  avoid  scheduling 
both  types  of  sessions  at  the  same  time.  Assessors  do  not 
participate  in  regularly  scheduled  supervision  and  consultation 
of  clinical  cases,  and  are  selectively  excluded  from  administrative 
meetings  in  which  discussion  of  the  treatment  process  might 
occur.  The  greatest  risk  to  these  barriers  is  the  patient  revealing 
the  method  of  treatment  delivery  to  the  assessor  at  one  of  the 


non-baseline  assessments.  As  such,  instructions  are  provided  to 
the  patient  at  the  outset  of  each  assessment  to  avoid  inadver¬ 
tently  revealing  condition  assignment.  If  a  patient's  treatment 
condition  is  inadvertently  revealed  to  an  outcomes  assessor,  the 
assessor  documents  the  occurrence  in  an  assessment  disposition 
note  to  allow  for  post-hoc  analysis. 

2.9.  Outcomes 

The  primary  outcome  variables  (continuous  measures)  are 
depressive  symptoms  measured  by  the  BD1-1I  and  hopelessness 
measured  by  the  BHS.  We  are  also  monitoring  patient  safety 
during  study  participation  in  order  to  establish  evidence  for  the 
safe  use  of  HBTMH.  Study  clinicians  document  safety  concerns 
and  record  any  adverse  safety  events  during  the  course  of 
treatment  on  the  Treatment  Session  Checklist.  Additional 
analyses  will  assess  treatment  group  differences  in  anxiety 
(BAI)  and  PTSD  (PCL-M)  symptoms,  patient  satisfaction  with 
and  attitudes  toward  treatment  (CSQ,  IASMHS),  quality  of  life 
(LS),  and  healthcare  utilization. 

2.10.  Statistical  methods 

We  are  using  a  non-inferiority  design  for  this  study 
because  we  expect  the  observed  efficacy  of  the  in-home 
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BATD  intervention  will  be  no  worse  than  that  observed  for 
in-person  BATD.  Non-inferiority  models  can  be  considered 
“a  one-sided  test  used  to  determine  if  a  novel  intervention  is  no 
worse  than  a  standard  intervention”  ([10],  p.  434).  The  non¬ 
inferiority  design  is  especially  useful  for  comparing  interventions 
that  have  been  modified  or  adapted  for  different  modes  of 
delivery  to  treatment  as  usual  [10],  Non-inferiority  trials  have 
also  been  previously  used  to  compare  telehealth  interventions 
to  conventional  in-person  care  (e.g.,  [7,23,29]).  Our  study  will 
test  the  primaiy  null  hypothesis  that  differences  between 
group  outcome  measures  for  patients  in  the  VCT-based  BATD 
condition  and  those  from  the  in-person  BATD  condition  will  be 
greater  than  a  set  threshold  or  margin  (labeled  A).  The  null 
hypothesis  will  be  rejected  and  thus  the  VCT  treatment 
considered  non-inferior  to  standard  in-person  treatment  if  the 
confidence  interval  calculated  around  the  treatment  difference 
falls  within  the  established  A 

2.10.1.  Power  analysis 

We  first  determined  the  non-inferiority  margin  based  on 
methodology  used  in  similar  studies  as  well  as  clinical 
considerations  [10,28].  A  0.5  standard  deviation  change  in 
scores  has  been  used  in  clinical  treatment  research  as  an 
indicator  of  clinically  significant  improvement  (e.g.,  [32,34]). 
This  margin  is  consistent  with  clinically  significant  change  in 
BDI-II  total  scores  and  standard  deviations  of  approximately 
10  points  in  both  military  [39]  and  civilian  [9]  samples.  From 
a  clinical  standpoint,  it  is  reasonable  to  consider  a  5  or  fewer 
point  change  in  BDI-II  scores  as  clinically  unimportant,  which 
also  aligns  with  the  0.5  standard  deviation  criteria  for 
significant  change  used  in  previous  research.  Thus,  we  set 
our  non-inferiority  margin  at  0.5  SD  and  used  a  2-sided  test 
with  a  90%  Cl  following  Mohr  et  al.'s  [22]  approach.  Power 
analyses  following  a  standardized  method  (variance  of  1) 
based  on  these  parameters  (Cohen's  d  of  0.5)  yielded  a 
minimum  sample  size  of  49  patients  in  each  treatment  group 
to  adequately  power  our  non-inferiority  analyses  assuming 
an  observed  difference  of  0  in  the  mean  efficacy  between  the 
two  study  groups.  Thus,  we  targeted  our  sample  size  for  120 
assuming  a  10%  rate  of  drop-out.  Although  we  will  pool  data 
from  the  two  participating  sites  for  analyses,  the  sample  sizes 
are  adequate  to  assess  any  potential  differences  in  demo¬ 
graphics  and  clinical  characteristics  between  the  two  sites. 

2.10.2.  Multilevel  model 

We  plan  to  follow  the  methodology  of  previous  non-inferiority 
studies  (i.e„  [7,22,24])  by  using  a  confidence  interval  method  for 
evaluating  the  difference  between  treatment  groups  and  pairing 
this  approach  with  an  intent-to-treat  (ITT)  analysis.  We  will  test 
the  primaiy  null  hypothesis  that  differences  in  outcome  measure 
scores  between  the  two  conditions  (in-home  vs.  in-office  BATD)  at 
the  post  assessment  time  point  will  be  greater  than  the  set 
clinically  relevant  threshold  or  margin  using  a  multilevel  (also 
referred  to  as  hierarchical  or  random  effects )  modeling  approach. 
The  primary  outcome  measure  is  BDI-II  score  and  secondary 
analyses  will  include  the  BHS.  We  will  include  both  individual  and 
sessions  as  units  of  the  analysis  with  patients  nested  within 
sessions.  The  baseline  values  for  the  outcome  measures  will  be 
accounted  for  in  the  model.  The  null  hypothesis  will  be  rejected  if 
the  upper  limit  of  the  Cl  calculated  around  the  difference  in  BDI-II 
scores  between  the  two  groups  falls  below  the  established  margin, 


suggesting  that  the  VCT  treatment  can  be  considered  non-inferior 
to  standard  in-person  treatment 

2.10.3.  Safety  and  feasibility 

Safety  will  primarily  be  evaluated  by  examining  the  rates 
of  safety  events  (e.g.,  activation  of  the  safety  management 
protocol  because  of  elevated  suicide  risk,  etc.)  during  the 
course  of  treatment.  Feasibility  will  be  assessed  by  examining 
the  frequency  and  types  of  technical  issues  that  occur  during 
the  course  of  treatment.  Patient  drop-out  rates,  levels  of 
patient  treatment  satisfaction  and  attitudes  toward  treat¬ 
ment  will  also  be  compared  between  the  treatment  condi¬ 
tions  to  help  evaluate  feasibility  of  HBTMH. 

3.  Discussion 

With  this  paper  we  have  presented  the  design,  methodology, 
and  protocol  of  a  clinical  trial  that  compares  in-office  to 
home-based  Behavioral  Activation  for  Depression  (BATD)  treat¬ 
ment  delivered  via  web-based  video  technology  for  service 
members  and  veterans  with  depression.  Recruitment  and 
assessment  phases  for  the  trial  are  underway  and  are  planned 
to  be  completed  by  the  end  of  2014.  This  trial  is  expected  to  yield 
important  data  that  can  help  guide  the  development  of 
treatment  guidelines  and  standards  of  care  (e.g.,  within  the 
Department  of  Defense  and  the  Veterans  Administration)  that 
aim  to  improve  access  to  quality  care  for  military  service 
members  and  veterans.  This  trial  will  also  demonstrate  the 
limitations  of  home-based  TMH  care  thereby  allowing  for  further 
refinement  of  safety  and  technical  procedures  to  maximize 
effectiveness  and  safety  of  this  modality  of  care. 

This  study  is  generating  important  infonnation  about  chal¬ 
lenges  and  best  practices  when  conducting  research  with  active 
duty  military  service  members  and  home-based  TMH.  Additional 
steps  must  be  taken  to  meet  requirements  of  multiple  review 
boards  and  oversight  committees,  which  may  take  more  time  and 
resources  than  is  typically  necessary  for  doing  similar  research  in 
non-military  settings.  There  is  additional  oversight  regarding 
reporting  of  adverse  events,  verifying  provider  credentials,  and 
accessing  equipment,  computing,  and  communications  systems 
within  the  militaiy.  Further,  patient  recruitment  strategies  must  be 
sensitive  to  varying  perceptions  regarding  stigma  associated  with 
mental  health  treatment  and  clinical  research  within  the  militaiy 
culture  and  community.  Retention  efforts  must  be  flexible  in  order 
to  accommodate  the  high  mobility  of  the  militaiy  population  given 
the  potential  for  relocation  and  deployment.  In  this  study, 
accommodations  have  been  made  to  conduct  follow-up  assess¬ 
ments  over  the  telephone  since  it  is  expected  that  some  patients 
may  have  relocated  (due  to  militaiy  assignment  or  discharge  from 
service)  during  the  period  between  completing  treatment  and  the 
3-month  follow-up  assessment.  We  hope  that  our  methods 
presented  here,  and  our  future  trial  results,  will  help  to  guide 
additional  research  regarding  home-based  TMH  treatments  in  the 
military  and  VA  settings. 

Factors  associated  with  an  active-duty  military  population 
also  impact  clinical  practices  and  feasibility  of  home-based 
treatments  in  the  MHS.  Working  within  this  system, 
clinicians  must  consider  specific  rules  regarding  the  protec¬ 
tion  of  privacy  and  confidentiality  that  may  be  different  than 
what  is  encountered  in  civilian  care  settings.  For  example, 
military  unit  commanders  are  authorized  to  verify  treatment 
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attendance  of  their  subordinates  and  clinicians  must  comply 
with  requirements  for  mandated  reporting  to  unit  chain  of 
commander  that  would  exceed  most  state  laws  (e.g.,  all  active 
substance  abuse,  any  suspected  incidence  of  domestic  violence). 
Also,  the  process  of  patient  safety  (e.g.,  suicide  risk  assessment) 
must  be  adapted,  as  we  have  done,  to  fit  local  requirements. 

We  are  also  collecting  treatment  adherence  data  with  our 
trial  that  will  help  us  to  determine  what  factors  including 
scheduling  may  influence  treatment  outcomes.  To  date,  we  have 
been  successful  in  working  with  patients  to  schedule  treatment 
sessions  during  day-time  hours.  While  home-based  options 
may  be  ideally  suited  for  patients  who  are  already  at  home 
(due  to  medical  leave,  unemployment,  etc.),  we  are  finding  that 
home-based  options  are  feasible  in  the  military  setting  when 
sessions  are  scheduled  in  the  morning  before  work  or  at  the  end 
of  the  day.  In  addition  to  being  convenient  for  patients,  time 
away  from  work  and  travel  costs  can  be  minimized  because  the 
service  member  does  not  have  to  leave  work  for  a  session  at  a 
clinic  and  then  return  back  to  work.  Home-based  TMH  care  may 
thus  be  an  ideal  solution  for  when  travel  to  a  military  treatment 
facility  or  clinic  is  not  feasible  or  if  there  is  limited  clinical  space 
near  where  the  patient  works  (i.e.,  in  remote  areas). 

In  conclusion,  home-based  mental  health  sen/ices  have  the 
potential  to  provide  effective  treatment  to  the  many  individ¬ 
uals  who  may  not  otherwise  pursue  mental  health  care,  either 
due  to  logistical  barriers  or  perceived  stigma  of  receiving  care. 
Home-based  treatment  options  may  be  particularly  useful 
in  addressing  the  aforementioned  barriers  to  care  and  can 
augment  current  treatment  services  provided  in  the  MHS  and 
VAHCS.  The  results  of  this  clinical  trial  will  provide  basic 
information  that  is  needed  to  inform  policy  decisions  regarding 
the  implementation  of  home-based  behavioral  health  care 
in  the  U.S.  military  and  further  expansion  in  other  settings 
including  the  VA  Health  System. 
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The  use  and  capabilities  of  telehealth  technologies  to  conduct  psychological  assessments  remotely  are 
expanding.  Clinical  practitioners  and  researchers  need  to  be  aware  of  what  influences  the  psychometric 
properties  of  telehealth-based  assessments  to  assure  optimal  and  competent  assessments.  The  purpose  of 
this  review  is  to  discuss  the  specific  factors  that  influence  the  validity  and  reliability  of  remote 
psychological  assessments  and  to  provide  best  practices  recommendations.  Specific  factors  discussed 
include  the  lack  of  physical  presence,  technological  issues,  patient  and  provider  acceptance  of  and 
comfort  with  technology,  and  procedural  issues.  Psychometric  data  regarding  telehealth-based  psycho¬ 
logical  assessment  and  limitations  to  these  data,  as  well  as  cultural,  ethical,  and  safety  considerations  are 
discussed.  The  information  presented  is  applicable  to  all  mental  health  professionals  who  conduct 
psychological  assessment  with  telehealth  technologies. 

Keywords:  psychological  assessment,  telehealth,  telemental  health,  video-conferencing,  mobile  devices 


The  use  of  telehealth  technologies  in  psychological  practice 
has  steadily  increased  over  the  last  decade  and  their  use  is 
expected  to  grow  substantially  in  the  years  ahead  (American 
Psychological  Association,  2010a;  Maheu,  Pulier,  McMenamin, 
&  Posen,  2012).  Psychological  assessment  is  an  integral  com¬ 
ponent  of  telemental  health  (TMH)  practice  and  is  necessary  for 
diagnostics,  screening,  symptom  monitoring,  and  evaluations  of 
treatment  progress  and  outcomes.  The  use  of  telehealth  tech¬ 
nologies  to  conduct  psychological  assessments  from  afar  can 
provide  convenience,  reduce  costs  (e.g.,  travel  avoidance),  and 
enable  access  to  assessment  services  when  they  are  otherwise 
unavailable. 

There  are  many  technologies  available  to  clinicians  who  are 
AQ:  2  engaged  in  TMH  practice.  These  include  traditional  telephones 
and  video-teleconferencing  (VTC)  equipment  for  synchronous 
(real-time)  communication  as  well  as  asynchronous  (store-and- 
forward)  technologies  such  as  fax  or  email  to  send  and  receive 
assessment  materials.  The  Internet  is  also  used  to  administer 
psychological  tests  and  measures  remotely  on  web  pages. 
Internet-based  testing  and  assessments  can  make  use  of 
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technology-enhancements  such  as  the  use  of  multimedia  content 
(i.e.,  pictures,  videos,  sounds,  etc.),  computer  adaptive  testing 
techniques,  and  automatic  scoring  and  interpretation  algorithms 
(see  Barak  &  Buchanan,  2004).  The  Internet  can  also  be  used  for 
VTC  by  using  personal  computers  (PCs)  and  off-the-shelf  web¬ 
cams,  which  may  be  an  affordable  and  highly  accessible  option  for 
home-based  TMH  (Luxton,  2013a).  More  recently,  smart  mobile 
devices  (i.e.,  smartphones  and  tablets)  have  emerged  as  a  way  to 
conduct  psychological  assessments.  Assessment  measures  can  now 
be  in  the  form  of  an  application  or  “app”  on  mobile  devices  or 
accessed  via  Internet  connectivity.  Traditional  interview  tech¬ 
niques  conducted  via  VTC  using  webcams  on  PCs  or  mobile 
devices  themselves  may  be  augmented  with  electronic  measures 
completed  on  the  device  with  data  uploaded  to  clinicians  for 
review  (Luxton,  McCann,  Bush,  Mishkind,  &  Reger,  2011). 

Given  the  numerous  available  telehealth  technologies  and  their 
increasing  use,  practitioners  who  use  them  need  to  be  cognizant  of 
the  factors  that  influence  the  psychometric  properties  of  psycho¬ 
logical  assessments  when  administered  via  those  technologies. 
Practitioners  also  need  to  know  whether  a  given  measure  or 
assessment  technique  is  appropriate  for  use  and  they  need  to  be 
familiar  with  the  proper  administration  procedures  in  order  to 
assure  competent  and  ethical  practice.  Our  purpose  with  this  article 
is  therefore  to  review  the  specific  issues  that  influence  the  validity 
and  reliability  of  telehealth-based  assessments  and  to  provide  best 
practice  recommendations  for  practitioners.  Although  we  focus 
primarily  on  psychological  assessment  and  evaluation  during  treat¬ 
ment  services  (i.e.,  diagnostic  and  symptom  assessment),  the  prin¬ 
ciples  and  procedures  that  we  discuss  are  pertinent  to  other  remote 
assessment  and  testing  applications  including  neuropsychological/ 
cognitive  testing,  forensic  risk  assessment,  and  occupational  test¬ 
ing. 
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Reliability  and  Validity  Considerations  and 
Recommendations 

Remote  Physical  Presence  and  Setting 

The  primary  and  most  obvious  difference  between  telehealth 
and  in-person  assessment  is  the  fact  that  the  patient  is  not  in  the 
same  room  as  the  clinician.  The  lack  of  in-person  presence  may 
influence  how  information  is  assessed  as  well  as  what  can  be 
assessed.  Nonverbal  information  is  useful  for  determining  the 
patient’s  emotional  state  and,  in  some  cases,  risk  behaviors.  For 
example,  olfactory  sensory  information  can  provide  clinically  rel¬ 
evant  information  regarding  hygiene  as  well  as  the  use  of  alcohol 
or  other  substances.  Body  posture,  facial  expressions,  body  lan¬ 
guage  (e.g.,  foot  tapping,  hand  wringing),  as  well  as  nonverbal 
emotional  responses  such  as  facial  flushing,  tearing  up,  and  direc¬ 
tion  of  eye-gaze,  also  provide  important  information.  The  obser¬ 
vation  of  psychomotor  and  other  medical  symptoms  are  also 
important  to  observe  during  psychological  assessments.  Further, 
the  observation  of  how  an  examinee  approaches  a  test  or  measure 
may  be  critical  for  making  an  accurate  assessment.  The  lack  of 
physical  presence,  however,  may  limit  the  range  of  information 
available  or  how  it  can  be  observed.  VTC  assessments  may  be 
influenced  by  camera  angle,  screen  size,  room  characteristics,  or 
other  technical  factors  (e.g.,  network  bandwidth  issues)  that  pro¬ 
hibit  the  observation  of  all  behaviors.  Further,  the  lack  of  physical 
presence  in  itself  may  influence  a  patient’s  clinical  presentation. 
For  example,  patients  who  are  socially  anxious  may  underreport 
symptom  severity  when  they  are  assessed  remotely  because  the 
fear-evoking  stimulus  (i.e.,  the  presence  of  the  assessor)  is  phys¬ 
ically  distant  (Grady  &  Melcer,  2005).  Also,  in  the  case  of  home- 
based  assessments,  symptoms  of  panic  disorder,  agoraphobia,  or 
the  hyper-arousal  symptom  cluster  of  PTSD  may  be  less  salient 
because  the  patient  is  able  to  avoid  situations  that  may  be  per¬ 
ceived  as  threatening,  such  as  driving  to  a  clinic  or  being  around 
strangers  in  a  busy  waiting  room. 

To  help  assure  the  validity  and  reliability  of  remote  assessment, 
it  is  first  necessary  to  make  sure  that  the  environmental  conditions 
at  the  remote  location  are  conducive  to  the  assessment  procedures. 
The  location  of  the  room  for  the  assessment  session  should  assure 
comfort  and  privacy.  The  assessment  space  should  be  large  enough 
for  the  patient  to  feel  comfortable  in  and  assessments  that  involve 
groups  and  family  interviews  will  require  a  space  that  is  large 
enough  to  accommodate  multiple  people  and,  for  some  applica¬ 
tions,  may  require  a  table  and  other  supplies  (Kramer,  Ayers, 
Mishkind,  &  Norem,  2011).  In  the  case  of  home-based  assessment, 
the  presence  of  roommates,  family  members,  pets,  unexpected 
phone  calls,  or  other  distractions  may  disrupt  the  assessment 
process.  It  is  therefore  important  for  the  practitioner  to  work  with 
the  patient  to  plan  for  and  schedule  sessions  during  a  time  that  is 
free  of  potential  disruptions.  These  considerations  are  particularly 
important  for  home-based  assessments  because  the  practitioner 
will  have  less  control  of  the  environment  than  they  may  have  in  an 
office  setting. 

Given  the  potential  limits  of  what  and  how  information  can  be 
collected  during  remote  assessments,  it  may  be  appropriate  to 
modify  typical  in-person  assessment  procedures.  However,  careful 
review  of  the  instructions  or  administration  manuals  for  measures 
and  tests  should  be  conducted  to  assure  that  procedures  or  envi¬ 


ronmental  conditions  for  standardized  administration  are  not  al¬ 
tered  in  a  way  that  threatens  the  reliability  and  validity  of  the 
assessment.  In  the  case  of  VTC-based  assessment,  it  may  be 
necessary  to  ask  a  patient  to  hold  a  paper-and-pencil  assessment 
(e.g.,  self-report  measures  or  therapy  homework)  up  to  the  video 
camera  for  viewing  or  to  use  larger  handwriting  because  of  small 
screen  size  or  poor  image  quality.  In  addition,  it  may  be  helpful  to 
ask  the  patient  to  read  their  responses  out  loud  in  scenarios  where 
synchronous  video  is  not  used  or  when  the  connection  quality  is 
inadequate.  When  nonverbal  information  is  useful  but  is  unavail¬ 
able  or  limited,  it  may  also  be  necessary  to  ask  additional  questions 
to  improve  the  accuracy  of  the  assessment.  For  example,  if  admin¬ 
istering  the  Hamilton  Rating  Scale  for  Depression  (Hamilton, 
1967),  it  may  be  appropriate  to  ask  the  patient  to  self-report 
symptoms  of  psychomotor  retardation  and  agitation  with  specific 
follow-up  questions  such  as  “Do  you  have  problems  sitting  still  for 
more  than  a  minute  or  two”  or  “Do  you  move  more  slowly  than 
your  coworkers?”. 

It  is  important  to  note  that  the  procedures  for  some  assessments 
may  not  lend  themselves  to  remote  administration  without  physical 
presence.  For  example,  the  Wechsler  Adult  Intelligence  Scale 
(WAIS-IV;  Wechsler,  2008)  involves  hands-on  interaction,  such 
as  administration  of  the  Block  Design,  Matrix  Reasoning,  and 
Visual  Puzzles  subtests  from  the  Perceptual  Reasoning  Index, 
which  would  be  inappropriate  and  impractical  to  administer  via 
VTC.  In  some  cases,  however,  it  may  be  feasible  to  administer 
assessments  remotely,  such  as  cognitive  function  testing  (see  Cul- 
lum,  Weiner,  Gehrmann,  &  Hynan,  2006),  whereby  an  on-site  staff 
member  administers  the  assessment  and  then  shares  the  results 
with  a  remote  clinician  who  scores  and  interprets  them.  Also,  some 
assessment  instruments,  such  as  the  Minnesota  Multiphasic  Per¬ 
sonality  Inventory  (MMPI)-2  or  WAIS-IV,  should  be  physically 
safeguarded  (not  made  openly  available  to  the  public)  to  assure  the 
validity  of  future  administrations.  It  is  therefore  important  for  the 
practitioner  to  consider  whether  remote  administration  of  assess¬ 
ment  materials  presents  a  risk  to  the  integrity  of  the  instrument 
(e.g.,  by  patients  being  able  to  print  items  at  home  or  share  them 
via  the  Internet,  etc.).  Practitioners  should  also  consider  whether 
there  is  an  increased  risk  for  dishonest  responses  (e.g.,  responses 
obtained  from  the  Internet  or  someone  else  taking  the  assessment) 
because  control  over  the  testing  environment  is  reduced  (Bu¬ 
chanan,  Johnson  &  Goldberg,  2005;  Reips,  2000). 

Technology  Issues 

There  are  several  technical  issues  associated  with  the  use  of  tele¬ 
health  technologies  that  may  influence  the  quality  of  telehealth-based 
assessments.  Eye  gaze  angle  is  the  angle  between  the  eye  and  the 
camera  and  the  eye  and  the  center  of  the  display  (Tam,  Cafazzo, 
Seto,  Salenieks,  &  Rossos,  2007).  A  potential  problem  when  using 
VTC  technology  is  that  users  often  make  eye  contact  with  the 
image  of  the  person  on  the  screen  rather  than  with  the  camera 
(Chen,  2002) — a  phenomenon  that  gives  the  appearance  that  one 
person  is  looking  down  or  away  from  the  other  person.  Eye  contact 
between  a  patient  and  a  clinician  is  important  because  it  provides 
visual  cues  to  which  the  participants  can  respond  (Grayson  & 
Monk,  2003;  Tam  et  al.,  2007).  Eye  contact  is  also  a  source  of 
clinical  information  that  is  useful  for  determining  the  presence  of 
psychological  states  or  particular  disorders  (e.g.,  autistic  disorder). 
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Interpretation  of  facial  expressions  and  affect  may  be  difficult 
when  eye  contact  is  misleading,  and  eye  gaze  angle  may  also 
influence  satisfaction  with  using  VTC  (Tam  et  al.,  2007).  Cameras 
should  be  positioned  in  a  way  that  allows  the  images  of  both 
parties  to  appear  straight-on  and  centered  in  their  respective  mon¬ 
itors  so  that  both  appear  to  speak  eye-to-eye  with  each  other 
(Kramer  et  ah,  2011).  Tam  et  al.  (2007)  pointed  out  that  improved 
eye  contact  can  be  realized  by  increasing  the  horizontal  distance  of 
participants  from  the  videoconferencing  unit.  Sometimes,  how¬ 
ever,  patients  may  shift  position  during  a  session,  or  the  camera 
may  be  accidently  be  shifted  from  the  optimal  angle.  It  may 
therefore  be  necessary  to  ask  the  patient  to  make  adjustments.  It  is 
also  recommended  to  check-in  with  the  patient  to  make  sure  they 
can  see  and  hear  clearly.  The  “picture  in  picture”  function  avail¬ 
able  on  many  VTC  devices  can  be  used  to  ensure  that  the  provider 
is  clearly  in  frame  as  well. 

Network  connection  quality  is  another  important  factor  that  can 
influence  assessment  capabilities  and  quality.  Connection  prob¬ 
lems  can  be  caused  by  a  variety  of  factors  such  as  low  quality 
equipment,  an  overloaded  computer  (e.g.,  too  many  programs 
running  at  one  time),  inadequate  bandwidth,  and  user  inexperience 
with  VTC  (Hyler,  Gangure,  &  Batchelder,  2005;  Jones,  Johnston, 
Reboussin,  &  McCall,  2001;  Luxton,  Mishkind,  Crumpton,  Ayers, 
&  Mysliwiec,  2012).  Jones  et  al.  (2001)  found  that  inadequate 
audio  quality  can  influence  the  ability  to  accurately  gather  infor¬ 
mation  from  the  patient.  For  example,  the  observation  of  vocal 
properties  (e.g.,  shakiness,  inflection,  and  tone),  as  well  as  whether 
an  individual  may  be  crying,  can  be  an  important  source  of 
information  regarding  emotional  states,  and  a  low-quality  audio 
connection  may  inhibit  observation  of  this  information.  It  is  also 
important  to  consider  that  technological  issues  (e.g.,  bandwidth 
limitations,  signal  drop-outs,  etc.)  may  influence  how  well  the 
patient  understands  the  clinician  (not  just  how  well  the  clinician 
understands  the  patient).  It  is  therefore  a  best  practice  to  test  the 
quality  of  the  connection  at  the  beginning  of  the  assessment 
session  and  to  check  in  with  the  patient  from  time  to  time  to  make 
sure  the  connection  quality  is  still  adequate.  Although  there  is 
some  guidance  available  regarding  minimum  recommended  band¬ 
width  requirements  for  TMH  (see  American  Telemedicine  Asso¬ 
ciation,  2009),  what  is  or  is  not  adequate  in  any  given  application 
will  depend  on  a  variety  of  factors,  including  requirements  for  the 
type  of  assessment,  environmental  conditions,  and  the  technology 
itself. 

Potential  distractions  caused  by  use  of  technology  (e.g.,  web 
cam,  personal  computer,  microphone,  mobile  device,  etc.)  may 
also  introduce  threats  to  the  validity  and  reliability  of  remote 
psychological  assessments.  For  example,  when  conducting  a  clin¬ 
ical  assessment  interview  over  a  web  cam,  the  patient  may  become 
distracted  by  inconsistent  connections,  error  messages,  or  other 
technical  anomalies  (Germain,  Marchand,  Bouchard,  Drouin,  & 
Guay,  2009;  Yoshino  et  al.,  2001).  Furthermore,  technical  mal¬ 
functions  during  telehealth  sessions  may  become  a  source  of 
frustration  for  patients  (Luxton,  Mishkind,  et  al.,  2012;  Luxton, 
Sirotin,  &  Mishkind.,  2010).  Persistent  technical  malfunctions  that 
occur  before  or  during  remote  assessment  sessions  may  therefore 
influence  motivation,  agreeableness,  and  adherence  to  assessment 
procedures.  It  is  thus  important  to  have  a  plan  to  resolve  technical 
malfunctions  by  expeditiously  troubleshooting  the  problem,  re¬ 


scheduling  the  session,  or  conducting  it  with  an  alternative  me¬ 
dium  (e.g.,  over  the  phone)  if  necessary. 

It  is  also  important  to  consider  potential  cognitive  and/or  sen¬ 
sory  deficits  that  patients  may  have  that  could  impair  their  ability 
to  use  telehealth  technology.  Technological  aides  (e.g.,  headsets, 
screen  magnification  devices,  speech  to  text  translation  software, 
etc.)  or  the  involvement  of  family  members  or  other  care  givers 
that  can  assist  may  be  appropriate.  Possible  fatigue  or  physical 
discomfort  caused  by  technology  use  (e.g.,  eye  strain  when  view¬ 
ing  computer  monitors)  should  also  be  evaluated  before  and  during 
the  assessment  process,  especially  during  lengthy  assessment  ses¬ 
sions. 

User  Acceptance 

Generally,  the  validity  of  any  psychological  assessment  is  mod¬ 
ulated  by  the  degree  to  which  the  person  being  assessed  accepts 
(i.e.,  is  willing  to  participate  in)  the  context  of  a  given  assessment 
including  the  setting  and  manner  in  which  the  assessment  is 
conducted  (Cronbach,  1970;  Elhai,  Sweet,  Guidotti  Breting,  & 
Kaloupek,  2012).  An  individual’s  acceptance  of  a  particular  type 
of  assessment  is  a  multifaceted  construct  that  depends  on  an 
individual’s  physical  and  emotional  state,  motivation,  attention, 
personality,  and  temperament.  Poor  acceptance  has  been  cited  as  a 
factor  that  reduces  compliance  and  the  motivation  to  engage  in 
mental  health  assessments  (Rogers,  2001).  Inadequate  acceptance 
of  TMH  by  either  the  patient  or  practitioner  can  therefore  be 
expected  to  have  a  negative  influence  on  the  validity  and  reliability 
of  psychological  assessments. 

Several  reviews  that  discuss  overall  acceptance  and  satisfaction 
with  TMH  provide  insight  into  the  factors  that  may  influence 
acceptance  of  telehealth-based  psychological  assessments.  For  ex¬ 
ample,  Modai  et  al.  (2006)  reported  that  patients  and  providers  are 
generally  satisfied  with  VTC  and  that  regular  use  of  VTC  im¬ 
proves  the  overall  degree  of  satisfaction  with  this  medium.  A 
review  by  Richardson,  Frueh,  Grubaugh,  Egede,  &  Elhai  (2009) 
showed  that  there  are  high  levels  of  user  satisfaction  and  accep¬ 
tance  with  TMH  across  diverse  clinical  populations  and  services. 
In  particular,  the  benefits  of  reduced  travel  time,  wait  times,  and 
lost  work  time,  as  well  as  greater  sense  of  personal  control  over 
sessions  were  specifically  associated  with  higher  satisfaction 
among  patients  (see  Hilty,  Nesbitt,  Kuenneth,  Cruz,  &  Hales, 
2007;  Simpson,  Bell,  Knox,  &  Britton,  2005).  These  benefits  may 
be  especially  important  when  considering  the  need  for  multiple 
visits  for  some  psychological  assessments  (i.e.,  initial  interview, 
assessment  battery  administration,  and  feedback/treatment  plan¬ 
ning). 

Technological  issues  may  also  play  an  important  part  in  the 
acceptability  of  telehealth-based  assessments  as  well  as  rapport 
between  the  patient  and  practitioner  (Glueck,  2013).  A  review  by 
Backhaus  et  al.  (2012)  found  that  patient  acceptability  of  VTC  is 
generally  on  par  with  the  acceptability  of  face-to-face  contact, 
although  the  most  common  areas  of  dissatisfaction  were  associated 
with  technical  difficulties  that  interrupted  sessions.  In  particular, 
problems  with  establishing  a  connection,  connection  speed,  sound 
echo/feedback,  and  inability  to  transmit  written  material  (e.g.,  a 
thought  journal  or  activity  log)  in  a  way  that  allowed  both  the 
patient  and  the  therapist  to  review  it  together  were  noted  (Cowain, 
2001;  Folen,  James,  Earles,  &  Andrasik,  2001).  Hyler,  Gangure, 
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and  Batchelder  (2005)  found  that  both  patients  and  providers 
preferred  in-person  assessment  when  compared  to  low-bandwidth 
VTC  assessment,  especially  when  detailed  observation  of  patients 
was  necessary;  however,  when  high-bandwidth  VTC  was  avail¬ 
able,  this  method  was  preferred  over  in-person  assessments.  Also, 
the  loss  or  distortion  of  nonverbal  behavior  and  other  patient 
characteristics  may  also  negatively  impact  a  clinician’ s  acceptabil¬ 
ity  of  telehealth-based  assessments  (Grady  &  Melcer,  2005).  So¬ 
lutions  to  these  technology-based  issues  are  available  (e.g.,  faster 
Internet  speeds,  head  sets,  adjunct  technology  such  as  fax/docu¬ 
ment  scanner)  but  come  with  financial  costs  that  may  limit  feasi¬ 
bility.  Overall,  it  is  important  for  practitioners  to  consider  that  the 
factors  that  may  influence  the  acceptability  of  telehealth-based 
psychological  assessments  may  not  be  consistent  across  all  assess¬ 
ment  sessions  or  settings. 

Cultural  Considerations 

As  with  in-person  psychological  assessment  and  testing,  prac¬ 
titioners  and  researchers  who  make  use  of  telehealth  technologies 
must  attend  to  a  broad  range  of  cultural  factors,  including  the 
patient’s  age,  technological  familiarity,  and  culture-specific  norms 
to  assure  valid  and  reliable  assessments.  For  example,  the  remote 
physical  presence  inherent  in  TMH  may  create  a  barrier  that 
reduces  a  patient’s  engagement  in  the  assessment  process,  espe¬ 
cially  among  members  of  cultures  or  groups  that  emphasize  inter¬ 
personal  connectedness  or  that  rely  heavily  on  nonverbal  interac¬ 
tions  (Nieves  &  Stack,  2007;  Savin,  Glueck,  Chardavoyne,  Yager, 
&  Novins,  2011).  An  essential  component  of  this  is  the  provider’s 
ability  to  make  use  of  whatever  nonverbal  communication  is 
available.  This  is  a  skill  that  is  essential  when  working  with  groups 
where  the  symptoms  of  mental  illness  may  be  minimized  or 
stigmatized  (e.g.,  Asian  and  Asian  American  populations,  military 
populations;  Yeung,  Hails,  Chang,  Trinh,  &  Fava,  2011).  Also, 
patients  that  are  less  comfortable  or  have  less  experience  with 
technology,  such  as  elderly  or  severely  impoverished  populations, 
may  display  a  more  drastic  discrepancy  between  in-person  and 
VTC  assessments  (Rohland,  Saleh,  Rohrer,  &  Romitti,  2000). 

When  working  with  specific  populations,  the  provision  of  TMH 
services,  including  psychological  assessment,  should  be  tailored  to 
the  needs,  resources,  and  technological  infrastructure  of  the  local 
community  (Brooks,  Spargo,  Yellowlees,  O’Neill,  &  Shore,  2013). 
Preliminary  work  suggests  that  the  customization  of  TMH  to  the 
needs  and  features  of  the  group  that  is  being  served  has  the 
potential  to  enhance  access  to  psychological  services  within  tradi¬ 
tionally  underserved  populations  (Dwight-Johnson  et  al.,  2011). 
Shore  et  al.  (2008)  have  also  demonstrated  that  structured  assess¬ 
ments  provided  via  VTC  were  as  acceptable  as  in-person  assess¬ 
ments  among  an  American  Indian  population  and  that  the  VTC  use 
did  not  influence  ratings  of  the  perceived  usability  of  the  assess¬ 
ment,  patient/provider  interaction,  or  overall  satisfaction.  Also, 
individual  backgrounds  may  present  a  strong  contextual  influence 
on  whether  and  how  technology  is  used  (Brooks,  Spargo,  Yellowlees, 
O’Neill,  &  Shore,  2013).  It  is  therefore  important  for  practitioners 
to  be  sensitive  to  the  capabilities  and  preferences  of  patients  during 
TMH  assessments  and  also  recognize  that  telehealth-based  assess¬ 
ments  may  not  be  appropriate  for  all  individuals.  The  provision  of 
a  brief  questionnaire  or  interview  survey  to  evaluate  previous 
experiences  and  preferences  regarding  technology  may  be  helpful. 


Ethical,  Privacy,  and  Safety  Considerations 

Attention  to  general  ethical  principles,  such  as  those  specified  in 
the  Ethical  Principles  of  Psychologists  and  Code  of  Conduct 
(American  Psychological  Association,  2010b),  is  necessary  during 
psychological  practice  whether  it  is  conducted  in-person  or  re¬ 
motely.  Standard  9,  Assessment,  specifically  addresses  standards 
for  psychological  assessment  and  these  too  apply  to  all  forms  and 
mediums  of  psychological  assessment.  There  are,  however,  aspects 
of  telehealth-based  assessments  that  require  additional  thought  in 
order  to  assure  ethical  practice  and  optimal  assessments.  For 
example,  it  would  be  inappropriate  practice  to  select,  develop,  or 
modify  assessment  instruments  or  alter  procedures  for  remote 
administration  without  evidence  of  sufficient  scientific  validation 
or  the  appropriate  disclosure  of  limitations.  It  is  therefore  neces¬ 
sary  for  practitioners  to  be  familiar  with  what  measures  or  tech¬ 
niques  are  supported  by  the  scientific  literature  before  using  them. 
Moreover,  the  assurance  of  patient  confidentiality  is  an  example  of 
ethical  practice  that  may  influence  the  validity  and  reliability  of 
psychological  assessments.  If  a  patient  does  not  feel  that  their 
privacy  is  respected  and  valued  by  practitioners,  the  patient  may  be 
less  willing  to  disclose  information  (Rogers,  2001). 

Both  physical  and  electronic  safeguards  should  be  used  to  assure 
confidentiality  during  remote  psychological  assessments.  For  ex¬ 
ample,  people  may  speak  louder  when  using  telehealth  technology 
than  when  in-person  and  electronic  speakers  may  amplify  sound 
significantly.  Thus,  audio  should  only  be  loud  enough  at  each  end 
so  that  both  the  patient  and  practitioner  can  be  easily  heard  but  not 
so  loud  that  the  TMH  session  can  be  overheard  by  people  outside 
the  room  (Kramer  et  al.,  2011).  In  the  case  of  home-based  TMH 
assessments,  practitioners  should  assess  whether  the  patient  has 
any  extra  concerns  about  their  privacy  (i.e.,  whether  family  mem¬ 
bers  or  others  may  overhear  the  assessment  session). 

Practitioners  conducting  psychological  assessments  with  tele¬ 
health  technology  also  need  to  be  cognizant  of  the  applicability  of 
the  Health  Insurance  Portability  and  Accountability  Act,  the 
Health  Information  Technology  for  Economic  and  Clinical  Health 
Act,  applicable  state  law,  and  local  privacy  and  security  require¬ 
ments.  The  American  Telemedicine  Association  provides  specific 
practice  standards  and  guidelines  regarding  this  topic  (American 
Telemedicine  Association,  2009,  2013).  Appropriate  disclosure  of 
safeguards  and  potential  risks  associated  with  privacy  and  elec¬ 
tronic  data  should  be  addressed  during  the  informed  consent  pro¬ 
cess.  As  pointed  out  by  Maheu  and  McMenamin  (2013)  however, 
the  informed  consent  process  or  an  agreement  with  patients  may 
not  be  adequate  in  all  situations,  in  all  states,  or  in  foreign  coun¬ 
tries.  Moreover,  the  diversity  in  the  types  of  technologies,  network 
infrastructures,  and  procedures  for  their  use  requires  careful  review 
of  data  security  risks  and  requirements  (see  Kramer,  Mishkind, 
Luxton,  &  Shore,  2013;  Luxton,  Kayl,  &  Mishkind,  2012). 
Whether  or  not  a  particular  technology  platform  or  application 
meets  standards  for  security  and  privacy  of  data  can  be  compli¬ 
cated  by  complex  issues  such  as  whether  and  how  digital  data  is 
stored  on  commercial  servers,  manufacturer  agreements  regarding 
ownership  of  transmitted  data,  and  other  potential  technical  risks  to 
data  security  and  privacy.  Consultation  with  applicable  legal  or 
regulatory  offices,  information  technology  system  administrators, 
equipment  and  software  manufacturers,  and  other  experienced 
experts  in  the  field  may  be  necessary  when  selecting  telehealth 
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platforms  or  when  uncertainties  regarding  particular  applications 
exist. 

It  is  also  important  to  consider  the  safety  of  the  patient  during 
TMH  assessment  sessions  and  to  have  a  safety  plan  in  place 
(Luxton,  O’Brien,  McCann,  &  Mishkind,  2012).  A  principal  con¬ 
cern  involves  what  to  do  if  a  patient  becomes  distressed  or  has  a 
medical  emergency  during  a  remote  assessment  session.  Safety 
plans  should  include  procedures  for  contacting  emergency  services 
in  the  patient’s  locale,  alternate  contact  methods  in  case  the  syn¬ 
chronous  telehealth  connection  is  lost  (e.g.,  backup  phone  contact), 
and  plans  for  resolving  technical  problems  (American  Telemedi¬ 
cine  Association,  2013;  Luxton,  O’Brien  et  al.,  2012).  The  iden¬ 
tification  and  involvement  of  a  local  collaborator,  such  as  a  family 
member  or  friend  of  the  patient  that  can  assist  with  on-site  tech¬ 
nical  problems  or  provide  support  to  a  patient  during  emergency 
situations  should  be  considered  (American  Telemedicine  Associ¬ 
ation.  2013;  Gros,  Veronee,  Strachan,  Ruggerio,  &  Acierno,  2011; 
Luxton,  O'Brien  et  al.,  2012).  The  use  of  a  collateral  person,  as 
well  as  overall  telehealth  assessment  procedures,  risks,  and  bene¬ 
fits  should  be  addressed  during  the  informed  consent  process. 

Patients  with  a  history  of  adverse  reactions  during  treatment 
(e.g.,  severe  panic  attacks),  or  those  who  are  at  high  risk  of  harm 
to  self  or  others  (e.g.,  family  members  in  the  case  of  home-based 
TMH),  may  not  be  appropriate  candidates  for  telehealth  services 
provided  to  clinically  unsupervised  settings  (Luxton,  O’Brien  et 
al.,  2012).  These  issues  should  also  be  considered  when  conduct¬ 
ing  remote  assessments,  especially  when  providing  assessment 
results.  As  noted  by  Pope  (1992),  the  form  of  assessment  results 
and  the  process  of  presenting  them  may  influence  how  patients 
interpret  their  meaning.  Given  these  concerns,  it  is  important  to 
consider  whether  providing  assessment  results  remotely  via  tele¬ 
health  technologies  is  appropriate  for  any  given  patient.  Prior  to 
engaging  in  remote  assessments,  review  of  the  patient’s  history 
and  potential  risks,  assessment  of  available  technologies  and  the 
patient’s  familiarity  with  them,  as  well  as  discussion  of  preferences 
regarding  engaging  in  remote  assessments  are  recommended.  Al¬ 
ternative  options  (e.g.,  in-person)  may  be  necessary  if  patients  are 
not  appropriate  candidates  for  telehealth-based  services  due  to 
safety  concerns,  clinical  contraindications,  technological  barriers, 
or  personal  preferences  (Luxton,  O'Brien,  et  al.,  2012;  Luxton, 
Sirotin,  &  Mishkind,  2010). 

Selecting  Assessment  Measures:  Psychometric 
Considerations 

It  is  important  for  practitioners  to  consider  that  even  if  an 
assessment  tool  has  been  shown  to  be  valid  and  reliable  in  the 
original  paper  form  or  in  one  particular  modality  (i.e.,  in-person 
interview  or  computer-based  administration)  it  does  not  necessar¬ 
ily  mean  that  the  measure  or  tool  will  be  valid  or  reliable  when 
conducted  remotely  via  telehealth  technologies.  Moreover,  even  if 
there  is  empirical  support  for  the  use  of  a  particular  measure  in  one 
telehealth  technology  medium,  such  as  on  an  Internet  web  page,  it 
does  not  necessarily  mean  that  it  will  be  valid  or  reliable  when 
transferred  to  another  medium,  such  as  a  mobile  device.  The 
differences  in  the  physical  format  of  these  mediums  and  proce¬ 
dures  for  use  may  influence  the  psychometric  properties  of  mea¬ 
sures  administered  through  them.  As  mentioned  previously  in  this 
article,  it  is  necessary  for  practitioners  to  be  familiar  with  the 


available  scientific  literature  regarding  a  measure  or  technique’s 
appropriateness  for  use. 

Several  published  reviews  provide  useful  information  regarding 
the  validity  and  reliability  of  remote  psychological  assessment  via 
various  telehealth  technologies.  For  example,  Hyler  et  al.  (2005) 
conducted  a  review  and  meta-analysis  that  included  14  studies  that 
compared  telepsychiatry  to  in-person  psychiatric  assessments.  Five 
studies  used  objective  assessment  measures,  two  studies  used 
subjective  measures  (i.e.,  satisfaction  measures),  and  seven  studies 
used  a  combination  of  objective  assessment  measures  and  subjec¬ 
tive  measures.  The  Brief  Psychiatric  Rating  Scale  (Overall  & 
Gorham,  1962)  was  the  most  common  assessment  instrument.  The 
meta-analysis  results  indicated  that  objective  telepsychiatry  assess¬ 
ments  were  similar  to  in-person  assessments  in  regard  to  diagnosis 
or  symptoms  assessment.  The  Telemental  Health  Standards  and 
Guidelines  Working  Group  (Grady  et  al.,  2011)  also  conducted  a 
review  of  published  data  regarding  the  psychometric  properties  of 
remote  telehealth-based  psychological  assessment.  They  noted 
several  studies  that  have  examined  psychological  assessment  via 
clinical  interviews  or  psychiatric  interviews  based  on  the  Struc¬ 
tured  Clinical  Interview  for  DSM  Disorders  (SCID;  Spitzer,  Wil¬ 
liams,  Gibbon,  &  First,  1990).  Two  studies  (Ruskin  et  al.,  1998; 

Shore,  Savin,  Orton,  Beals,  &  Manson,  2007)  demonstrated  high 
reliability  in  the  administration  of  the  SCID.  Comparability  be¬ 
tween  face-to-face  and  VTC  is  also  demonstrated  for  the  Hamilton 
Depression  Rating  Scale  for  depression  (Kobak,  2004;  Kobak, 
Williams,  &  Engelhardt,  2008).  Backhaus  and  colleagues  (2012) 
conducted  a  systematic  review  of  the  research  on  psychotherapy 
using  VTC  and  reported  that  69%  of  the  42  studies  that  they 
reviewed  used  a  well-accepted  psychometrically  validated  (in- 
person)  standardized  measure  for  treatment  outcomes.  Of  these, 
the  most  common  assessment  (24%  of  the  42  studies)  was  the 
Beck  Depression  Inventory  (BDI-II;  Beck,  Steer,  &  Brown,  1996). 

Grady  et  al.  also  noted  that  there  are  not  any  psychometric  data 
available  regarding  projective  testing  over  VTC. 

Grady  et  al.  also  noted  evidence  that  demonstrates  the  feasibility 
of  remote  neuropsychological  assessment  (Hildebrand,  Chow, 
Williams,  Nelson,  &  Wass,  2004;  Saligari  et  al.,  2002)  and  com¬ 
parability  of  scores  between  remote  and  in-person  assessment 
(Cullum  et  al.,  2006;  Loh,  Donaldson,  Flicker,  Maher,  &  Gold- 
swain,  2007),  as  well  as  some  research  that  has  demonstrated 
differences  on  test  scores  (Ball.  Tyrrel,  &  Long,  1999;  Loh  et  al., 

2004;  Montani  et  al.,  1996).  Cognitive  assessments  that  have  been 
examined  and  validated  include  the  cognitive  section  of  the  Cam¬ 
bridge  Mental  Disorders  of  the  Elderly  Examination  (Ball  & 
Puffett,  1998),  the  Mini-Mental  State  Examination  (Grob,  Wein- 
trau,  Sayles,  Raskin,  &  Ruskin,  2001),  the  National  Adult  Reading 
Test,  and  the  Adult  Memory  and  Information  Processing  Battery 
(Tschirch,  Walker,  &  Calvacca,  2006).  The  development  of  new 
norms  has  been  recommended  so  that  the  thresholds  used  for 
impairment  are  valid  when  compared  with  face-to-face  adminis-  AQ:3 
tration  (Grady  et  al.,  2011;  Kirkwood,  Peck,  &  Bennie,  2000). 

Evidence  of  equivalence  as  well  as  differences  between 
Internet-based  questionnaire  assessments  and  paper-and-pencil  ad¬ 
ministrations  of  standardized  measures  has  also  been  reported  in 
the  literature  (Barak  &  English,  2002;  Barak,  Hen,  Boniel-Nissim, 

&  Shapira,  2008;  Buchanan,  2002;  Naglieri  et  al.,  2004).  For 
example,  evaluation  of  online  versions  of  BDI-II  have  shown  that 
psychometric  properties  differ  across  testing  modalities  of  psycho- 
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metrically  validated  assessments,  even  when  comparisons  are 
made  between  equivalent  samples,  and  that  online  BDI-II  scores 
tended  to  be  higher  (Glaze  &  Cox,  1991;  Peterson,  Johannsson,  & 
Carlsson,  1996;  Schulenberg  &  Yutrzenka,  1999).  Also,  some 
evaluations  of  personality  inventories  administered  online  have 
shown  differences  in  item  loadings  compared  to  paper-and-pencil 
versions  (Buchanan,  2001;  Buchanan,  Johnson,  &  Goldberg, 
AQ:  4  2005).  These  differences  may  be  due  to  the  lessened  impact  of 
social  desirability,  an  increase  of  self-disclosure  in  online  assess¬ 
ment,  or  as  the  result  of  the  technology-based  presentation  itself 
(Buchanan,  2003). 

In  sum,  the  literature  base  regarding  the  psychometric  properties 
of  telehealth-based  assessments  is  growing;  however,  there  are 
gaps  in  the  literature  that  practitioners  should  consider  when 
selecting  particular  assessment  instruments  and  mediums.  In  par¬ 
ticular,  the  vast  majority  of  available  measures  and  assessment 
tools  are  based  on  norms  that  were  established  by  employing 
traditional  in-person  procedures.  The  reevaluation  of  these  tools 
with  diverse  populations,  clinical  presentations,  and  telehealth 
mediums  is  necessary  to  assure  the  validity  of  assessments  con¬ 
ducted  via  telehealth  technologies.  It  is  critical  for  practitioners  to 
be  cognizant  of  assessment  measure  limitations  and  to  appropri¬ 
ately  disclose  and  document  them  in  their  practice.  Keeping  up 
with  the  scientific  literature  as  well  as  publications  by  organiza¬ 
tions  such  as  the  APA  and  American  Telemedicine  Association  is 
recommended. 

Discussion 

The  validity  and  reliability  of  psychological  assessments  con¬ 
ducted  via  telehealth  technology  is  influenced  by  factors  that  are 
both  common  to  in-person  assessment  and  unique  to  telehealth- 
based  assessments.  Although  the  psychometric  characteristics  and 
standardized  procedures  of  traditional  in-person  psychological  as¬ 
sessments  provide  useful  information  about  how  they  may  trans¬ 
late  to  other  mediums,  practitioners  would  be  remiss  to  simply 
assume  equivalency  between  in-person  and  remote  administration 
of  psychological  assessments.  It  is  therefore  important  for  practi¬ 
tioners  who  already  use  or  are  considering  the  use  of  telehealth 
technologies  to  be  familiar  with  these  factors  and  to  use  appropri¬ 
ate  administration  techniques  when  conducting  remote  psycholog¬ 
ical  assessments.  It  is  feasible  to  assure  reliable  and  valid  remote 
psychological  assessments  with  appropriate  knowledge,  prepara¬ 
tion,  and  practice. 

The  use  of  telehealth  technologies  for  remote  psychological 
assessment  has  several  benefits  for  both  patients  and  practitio¬ 
ners.  For  example,  telehealth-based  assessments  allow  practi¬ 
tioners  to  conveniently  monitor  symptoms  and  other  health 
variables  between  in-person  or  telehealth  treatment  sessions.  Fur¬ 
ther,  telehealth-based  psychological  assessment  may  improve  care 
satisfaction  and  overall  health  outcomes  by  providing  services  that 
are  specialized  for  the  patient’s  needs.  In  particular,  telehealth 
technologies  may  provide  access  to  clinical  specialty  assessments 
(e.g.,  neuropsychological  assessments)  that  are  not  available  in  the 
patient’s  locale.  Telehealth-based  psychological  assessment  may 
also  increase  access  to  services  among  patients  who  speak  differ¬ 
ent  languages.  For  example,  a  non-English  speaking  patient  could 
engage  in  mental  health  assessment  and  treatment  with  a  clinician 
who  speaks  their  native  language,  regardless  of  physical  location, 


which  may  minimize  potential  misunderstanding  and  misdiagnosis 
of  a  patient’s  symptom  report  (Yeung  et  al.,  2011).  This  would  also 
be  true  for  the  use  of  American  Sign  Language  and  would  remove 
the  need  for  a  third-party  translator  who  may  unintentionally 
change  the  meaning  of  a  communication  during  the  translation 
process.  The  use  of  telehealth  technologies  also  allows  patients  to 
connect  with  providers  that  are  trained  in  specialized  assessments 
and  who  also  have  experience  working  with  particular  cultural 
groups  (e.g.,  military  culture,  the  elderly,  specific  ethnic  groups, 
etc.). 

Current  and  emerging  technologies  not  only  allow  remote  ad¬ 
ministration  of  traditional  assessments  but  may  also  offer  new  or 
improved  capabilities  and  methods  of  assessment.  In  particular,  the 
growing  field  of  mobile  device  apps  has  created  opportunities  for 
self-care  assessments  and  symptom  screening  that  were  not  pos¬ 
sible  just  a  decade  ago  (Luxton,  McCann  et  al.,  2011).  Assessment 
apps  on  smartphones  and  tablet  PCs  can  also  be  useful  for  mea¬ 
suring  the  dynamic  characteristics  of  a  person.  For  instance,  sub¬ 
jective  mood  or  anxiety  levels  can  be  tracked  in  real-time  and  data 
from  bio-feedback  equipment  can  be  tracked  and  analyzed  re¬ 
motely.  The  small  size  and  touch  screen  features  of  smartphones 
and  tablet  PC  devices  are  factors  that  may  influence  the  psycho¬ 
metric  characteristics  of  assessments  provided  on  these  devices. 
Preliminary  data,  however,  have  suggested  that  these  devices  may 
be  a  feasible  platform  for  assessments  that  is  comparable  to  paper- 
and-pencil  and  computer-based  assessments  (Bush,  Skopp,  Smo- 
lenski,  Crumpton  &  Fairall,  in  press).  More  research  is  needed, 
however,  regarding  the  psychometric  properties  of  psychological 
assessments  administered  via  mobile  devices. 

Assessments  conducted  with  computer-simulated  virtual  reality 
environments  are  another  emerging  capability  (Holloway  &  Reger, 

2012;  Parsons,  Silva,  Pair,  &  Rizzo,  2008;  Riva,  Wiederhold,  & 
Molinari,  1998).  Assessments  can  be  built  into  the  virtual  envi¬ 
ronment  so  that  measures  appear  virtually  while  a  patient  is  in  the 
virtual  environment  or  to  simulate  real-world  conditions  that  are 
useful  for  the  assessment  of  particular  variables.  For  example, 
virtual  environments  have  been  tested  as  a  way  to  create  environ¬ 
mental  or  social  cues  for  assessment  of  emotional  and  behavioral 
responses  among  patients  being  treated  for  addiction  or  anxiety 
related  behaviors  (see  Bordnick,  Carter,  &  Traylor,  2011).  The  AQ:5 
application  of  artificial  intelligence  technologies  to  conduct  clini¬ 
cal  interviews,  psychological  assessments,  and  evaluations  is  also 
a  promising  area  (Luxton,  2013b).  Virtual  intelligent  agents  capa¬ 
ble  of  human-like  social  interaction  can  be  designed  to  conduct 
clinical  interviews,  analyze  results,  and  provide  feedback  to  pa¬ 
tients.  These  types  of  systems  have  already  been  developed  for 
clinical  training  and  some  treatment  services  (DeAngelis,  2012; 
Parsons.  Kenny,  et  al.,  2008).  Artificial  intelligence-enabled  tech¬ 
nologies  that  use  advanced  sensing  and  language  processing  ca¬ 
pabilities  are  also  being  developed  to  assess  physiological  and 
psychological  variables.  These  advances  in  technology  have  the 
potential  to  increase  the  reliability  and  validity  of  psychological 
assessment,  improve  clinical  care,  and  reduce  costs  for  both  pa¬ 
tients  and  practitioners. 

In  conclusion,  the  use  of  telehealth  technologies  provide  an 
opportunity  for  psychologists  and  other  health  care  professionals 
to  expand  the  capabilities  of  their  practice,  provide  quality  ser¬ 
vices,  and  meet  the  health  care  needs  of  care  seekers.  The  in¬ 
creased  user  demand  for  technology  as  well  as  the  continued 
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growth  of  TMH  services  will  push  the  need  for  telehealth-based 
psychological  assessments.  The  adherence  to  best  practices  and 
competencies  for  psychological  assessment  via  telehealth  technol¬ 
ogies  is  the  responsibility  of  psychologists  and  others  who  provide 
such  services.  Practitioners  must  remain  familiar  with  available 
research  and  guidelines  before  engaging  in  remote  assessments. 
Moreover,  practitioners  must  consider  the  applicability  to  specific 
populations  and  appropriateness  of  any  assessment  measure  or 
technique  on  a  case-by-case  basis.  Ultimately,  the  decision  to 
conduct  psychological  assessments  from  afar  should  depend  on 
both  the  practitioner’s  and  patient’s  comfort  level  with  the  process. 
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Background:  Home-based  tele-behavioral  health  (HBTBH)  care  has  the  potential  to  provide  US  military  Service 
members  with  increased  access  to  care,  convenience,  and  privacy.  While  the  Veteran’s  Health  Administration 
(VHA)  has  already  sanctioned  HBTBH  services,  home-based  care  is  not  presently  a  standard  of  care  in  the 
Military  Health  System  (MHS).  The  safety,  feasibility,  and  effectiveness  of  HBTBH  needs  to  be  evaluated  in 
the  military  setting  before  it  can  be  considered  for  adoption  across  the  MHS.  This  presentation  will  report  the 
results  of  the  first  randomized  controlled  trial  of  a  tele-behavioral  health  treatment  delivered  via 
videoconferencing  technology  (VCT)  directly  to  the  homes  of  U.S.  Service  members. 

Methods:  Participants  meeting  DSM-IV  criteria  for  depression  were  recruited  from  outpatient  behavioral  health 
programs  at  a  large  regional  military  treatment  facility  and  VHA  medical  center.  Following  informed  consent, 
participants  (n=121)  were  randomized  to  receive  Behavioral  Activation  Treatment  for  Depression  either  in- 
person  or  in-home  via  a  laptop  and  webcam.  Both  groups  received  the  same  manualized  treatment  protocol 
which  included  eight  weekly  one-hour  sessions.  Clinical  assessments  occurred  at  baseline,  4-week  midpoint,  8- 
week  treatment  completion,  and  3-month  follow-up  time  points.  Depression,  hopelessness,  anxiety  symptoms, 
treatment  seeking  attitudes,  treatment  satisfaction,  and  occurrence  of  any  events  involving  initiation  of  a  safety 
protocol  were  assessed. 

Results:  For  participants  in  the  in-person  condition,  Beck  Hopelessness  Scale  (BHS)  scores  decreased  by  an 
average  of  6.20  points  (95%  Cl  =  -7.74,  -4.67)  and  Beck  Depression  Inventory  (BDI-II)  scores  decreased  by  an 
average  of  17.64  points  (95%  Cl  =  -20.90,  -14.38)  at  post  treatment  assessment.  Among  in-home  participants, 
BHS  scores  decreased  by  an  average  of  3.90  points  (95%  Cl  =  -5. 1 1,  -2.70)  and  BDI-II  scores  decreased  by  an 
average  of  13.39  points  (95%  Cl  =  -16. 17,  -10.60)  at  post  treatment.  A  linear  mixed  effects  regression  model 
was  used  to  test  whether  the  upper  bound  of  a  90%  confidence  interval  exceeded  the  non-inferiority  margin  of 
0.50  at  the  post  treatment  assessment  period.  The  confidence  interval  exceeded  the  margin  for  both  the  BHS  and 
BDI-II  scores,  indicating  that  we  could  not  reject  the  null  hypotheses  of  inferiority  beyond  the  defined  margin. 
Using  a  two-tailed  95%  confidence  interval,  the  in-home  condition  demonstrated  inferiority  to  the  in-person 
condition  on  the  BHS,  but  the  interval  for  the  inferiority  of  BDI-II  included  zero.  Patient  safety  was  maintained 
effectively  in  both  groups  throughout  the  trial  and  favorable  attitudes  toward-  and  satisfaction  with-  the 
treatment  did  not  differ  between  the  two  treatment  groups. 

Conclusions:  This  study  represents  the  first  RCT  of  home-based  tele-behavioral  health  care  conducted  in  the  US 
military  setting.  Based  on  the  data,  delivering  psychotherapy  for  depression  in  both  a  face-to-face  and  in-home 
medium  reduced  depression  and  hopelessness  scores.  However,  the  comparison  of  these  modalities  did  not 
support  the  non-inferiority  of  HBTBH  compared  to  standard  in-person  services.  The  observed  results  regarding 
overall  safety  and  treatment  satisfaction  support  home-based  tele-behavioral  healthcare  as  a  potentially  viable 
option  for  meeting  the  health  care  needs  of  U.S.  military  and  VHA  health  care  beneficiaries  that  don’t  have 
access  to-  or  are  unwilling  to  seek  out-  traditional  services. 
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Home-based  telemental  health  (HBTMH)  has  several  important  benefits  for  both  patients  and  clinical 
practitioners  including  improved  access  to  services,  convenience,  flexibility,  and  potential  cost  savings. 
HBTMH  also  has  the  potential  to  offer  additional  clinical  benefits  that  are  not  realized  with  traditional 
in-office  alternatives.  Through  a  review  of  the  empirical  literature,  this  article  presents  and  evaluates 
evidence  of  the  clinical  benefits  and  limitations  of  HBTMH.  Particular  topics  include  treatment  atten¬ 
dance  and  satisfaction,  social  support,  access  to  contextual  information,  patient  and  practitioner  safety, 
and  concerns  about  privacy  and  stigma.  By  making  use  of  commonly  available  communication  technol¬ 
ogies.  HBTMH  affords  opportunities  to  bridge  gaps  in  care  to  meet  current  and  future  mental  health  care 
needs. 

Keywords:  telemental  health,  home-based,  telehealth,  telemedicine,  telecommunications 


Home-based  telemental  health  (HBTMH)  is  the  delivery  of 
mental  health  services  directly  to  a  patient’s  home  via  the  use  of 
telecommunications  technology.  Current  technologies  used  to  fa¬ 
cilitate  HBTMH  include,  but  are  not  limited  to,  synchronous 
modalities  such  as  telephone  and  Internet-based  video  teleconfer¬ 
encing  (VTC)  and  asynchronous  (store-and-forward)  technologies 
such  as  e-mail  and  text  messaging  (Hilty  et  al.,  2013;  Hilty  et  al., 
2006).  Informational  Internet  sites  or  Web-based  interactive  pro- 
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grams  can  also  be  used  in  HBTMH  (Rizzo  et  al.,  2011).  Mobile 
devices  such  as  smartphones  and  tablets  have  also  recently 
emerged  as  a  method  to  provide  care  via  synchronous  video  or 
mental  health  “apps”  (Luxton,  McCann,  Bush,  Mishkind,  &  Reger, 
2011). 

Treatments  delivered  via  HBTMH  have  been  used  to  address 
diverse  mental  health  conditions  including  bipolar  disorder,  sub¬ 
stance  abuse,  depression,  obsessive-compulsive  disorder,  post- 
traumatic  stress  disorder  (PTSD),  schizophrenia,  and  panic  disor¬ 
der  (Bensink,  Hailey,  &  Wootton,  2006;  Brand  &  McKay,  2012; 
DelliFraine  &  Dansky,  2008).  Treatments  for  depression  and  anx¬ 
iety  delivered  via  video-based  HBTMH  have  garnered  the  most 
extensive  study  (e.g.,  Choi,  Hegel,  Marinucci,  Sirrianni,  &  Bruce, 
2013;  Gros  et  al.,  2011;  Strachan  et  al.,  2012)  and  have  primarily 
examined  the  application  of  cognitive- behavioral  therapies 
(CBT),  including  those  with  exposure-based  components  such  as 
prolonged  exposure  (e.g.,  Strachan  et  al.,  2012)  and  exposure  and 
response  prevention  (e.g.,  Brand  &  McKay,  2012).  Generally, 
clinical  outcomes  studies  suggest  that  HBTMH  is  both  a  feasible 
and  effective  treatment  option  and  that  both  in-home  and  in-person 
treatments  demonstrate  comparable  effects  with  regard  to  both  the 
magnitude  and  the  direction  of  change  over  the  course  of  specific 
cognitive- behavioral  treatment  protocols  (Gros,  Yoder,  Tuerk, 
Lozano,  &  Aciemo,  2011;  Gros  et  al.,  2012;  Rabinowitz,  Brennan, 
Chumbler,  Kobb,  &  Yellowlees,  2008;  Strachan  et  al.,  2012).  The 
maintenance  of  clinical  gains  for  periods  ranging  from  3  months  to 
2  years  have  also  been  reported  among  individuals  with  serious 
mental  illness  diagnoses  as  well  as  home-bound  older  adults  with 
depression  (Choi  et  al.,  2012;  D’Souza,  2002).  There  is  also 
evidence  that  telemental  health,  including  HBTMH,  can  help  pre¬ 
vent  inpatient  rehospitalization,  reduce  the  number  of  days  spent  in 
psychiatric  hospitals,  and  improve  overall  treatment  satisfaction 
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and  compliance  (D’ Souza,  2002;  Godleski,  Darkins,  &  Peters, 

2012). 

In  sum,  evidence  is  accumulating  to  suggest  that  the  degree  of 
clinical  effectiveness  of  HBTMH  treatments  is  comparable  with 
the  same  treatments  delivered  in  conventional  in-office  settings. 
There  are  also  unique  aspects  of  HBTMH  that  may  provide  addi¬ 
tional  clinical  benefits  and  enhancements  that  are  not  realized 
in  traditional  in-office  care  settings.  Our  goal  with  this  article  is  to 
highlight  and  review  the  evidence  of  the  additional  benefits  that 
have  been  given  less  attention  in  the  literature.  Although  we 
recognize  that  self-directed  mental  health  services  accessed  via  the 
Internet  at  home  are  a  form  of  home-based  treatment,  our  review 
is  focused  on  clinical  HBTMH  treatments  that  are  provided  di¬ 
rectly  to  the  patient,  in  real  time,  by  trained  mental  health  profes¬ 
sionals  (e.g.,  psychologists,  psychiatrists,  social  workers,  psychi¬ 
atric  nurses,  etc.).  Our  focus  is  primarily  on  telepractice  with 
mental  health  populations;  however,  some  of  the  benefits  that  we 
review  also  pertain  to  behavioral  medicine  applications  such  as 
pain  management,  sleep  disorder  treatment  and  diet/exercise  coun¬ 
seling.  Our  aim  is  to  provide  information  that  is  useful  to  practi¬ 
tioners  and  researchers  who  have  an  interest  in  HBTMH. 

Treatment  Attendance  and  Satisfaction 

One  of  the  principal  benefits  of  HBTMH  is  its  potential  to 
improve  treatment  attendance  and  satisfaction  which  can  lead  to 
more  positive  treatment  outcomes.  The  benefits  of  reduced  travel, 
less  time  off  work,  shorter  appointment  wait-times,  and  greater 
personal  control  are  frequently  cited  as  advantages  of  telehealth- 
based  care  over  in-person  care  (Hilty,  Nesbitt,  Kuenneth,  Cruz,  & 
Hales,  2007;  Simpson,  Bell,  Knox,  &  Britton,  2005).  These  ben¬ 
efits  are  especially  salient  with  HBTMH  because  of  the  cost- 
avoidance  benefit  of  not  having  to  travel  to  a  clinic  or  arrange  for 
childcare,  for  example. 

Another  benefit  of  providing  a  HBTMH  option  to  patients  is  that 
it  may  communicate  that  the  practical  concerns  the  patient  has 
about  seeking  treatment  (e.g.,  cost,  travel  time,  time  off  work,  etc.) 
are  valid.  This  idea  is  supported  by  research  that  has  shown  that  the 
validation  of  patient  concerns  is  associated  with  adherence  to  pain 
management  treatment  (Linton,  Boersma,  Vangronsveld,  &  Fruz- 
zetti,  2012).  This  patient-centered  validation  of  the  patient’s  needs 
on  the  part  of  the  practitioner  may  have  important  implications  for 
a  patient’s  willingness  to  seek  and  complete  treatment.  This  is 
consistent  with  a  study  that  showed  that  switching  to  HBTMH 
increased  both  medication  compliance  and  appointment  attendance 
among  adolescent  patients  who  were  noncompliant  with  in-person 
services  (Hommel,  Hente,  Herzer,  Ingerski,  &  Denson,  2013). 

HBTMH  may  also  be  a  preferred  option  for  some  patients  with 
anxiety  conditions  (e.g.,  social  anxiety,  panic  disorder  with  ago¬ 
raphobia,  specific  phobia)  who  are  contemplating  seeking  care  but 
who  are  concerned  about  confronting  anxiety  provoking  stimuli 
(e.g.,  crowds,  driving,  etc.).  Patients  who  are  less  able  to  tolerate 
anxiety  may  be  less  likely  to  seek  or  adhere  to  conventional 
in-office  care.  A  HBTMH  option  creates  an  opportunity  to  discuss 
treatment  rationale  and  provide  evidenced-based  treatments  to 
individuals  who  find  it  too  distressing  to  attend  treatment  in- 
person,  at  least  initially.  It  is  important  to  note  that  although 
HBTMH  provides  an  option  for  patients  who  may  not  otherwise 
attend  treatment  in  an  office  or  other  clinical  setting;  it  may  fail  to 


provide  the  benefit  of  naturalistic  behavioral  activation  and  expo¬ 
sure  that  would  occur  if  the  patient  was  to  attend  treatment  in 
person.  The  comfort  and  conveniences  of  staying  at  home  may 
inadvertently  minimize  a  patient’s  contact  with  beneficial  pro¬ 
cesses  such  as  attending  to  personal  hygiene,  leaving  the  house, 
engaging  in  social  interactions,  and  self-exposure  to  anxiety  pro¬ 
voking  stimuli  that  are  often  required  for  in-person  care.  It  is  thus 
important  for  practitioners  to  consider  and  work  with  patients  to 
find  opportunities  to  engage  in  therapeutic  and  exposure-based 
behaviors  during  the  time  between  treatment  sessions. 

Published  data  regarding  satisfaction  specific  to  HBTMH  is 
limited  at  this  time;  however,  the  broader  TMH  literature  does 
provide  evidence  of  patient  satisfaction  with  treatments  provided 
via  VTC  that  is  comparable  with  in-person  treatment  (Richardson 
et  al.,  2009).  Low-quality  VTC  software,  however,  can  have  neg¬ 
ative  effects  on  satisfaction  due  to  the  disruption  that  low  band¬ 
width  connections  can  have  on  the  continuity  of  VTC  image 
quality  and  fluidness  of  the  interaction  (Hyler,  Gangure,  &  Batch- 
elder,  2005).  Moreover,  a  perceived  lack  of  technical  savvy  and 
insufficient  experience  with  computer  equipment  has  been  cited  by 
potential  HBTMH  patients  as  a  reason  why  they  choose  to  avoid 
remote  care,  suggesting  that  fear  about  the  technological  aspects  of 
home-based  care  may  be  an  important  barrier  to  the  acceptance  of 
this  treatment  modality  (Shore,  Savin,  Novins,  &  Manson,  2006; 
Starling  &  Foley,  2006).  There  is  evidence,  however,  that  with 
minimal  training  HBTMH  patients  can  leant  to  use  telehealth 
equipment  and  software  effectively  in  relatively  short  amounts  of 
time  (Bischoff  et  al.,  2004;  Gabrielian  et  al.,  2013;  Shore  et  al., 
2006).  Cooperative  problem  solving  of  technical  issues  during 
treatment  may  also  have  several  therapeutic  benefits.  For  example, 
collaborative  set-up  or  troubleshooting  of  equipment  may  facilitate 
initial  and  sustained  rapport,  facilitate  collaborative  problem  solv¬ 
ing,  and  build  self-confidence. 

Practitioners  of  behavioral  health  care  also  report  overall  posi¬ 
tive  attitudes  and  levels  of  satisfaction  with  the  use  of  TMH, 
especially  in  terms  of  its  ability  to  improve  access  to  care.  This  is 
especially  true  among  those  who  have  had  direct  experience  using 
TMH  in  their  practice  (Simms,  Gibson,  &  O’Donnell,  2011). 
Indeed,  a  practitioner’s  regular  use  of  VCT  and  their  ratings  of  user 
satisfaction  are  positively  correlated  (Modai  et  al.,  2006)  which 
suggests  that  satisfaction  increases  as  experience  is  gained. 

Social  Support  and  Connectedness 

Social  support  is  a  key  factor  in  the  development,  severity,  and 
recovery  from  psychopathology  (e.g.,  Cohen  &  Wills,  1985;  Sara- 
son  &  Sarason,  1982).  Similarly,  the  interpersonal  connection  and 
empathic  support  present  in  psychotherapeutic  interventions  is  a 
meaningful  component  of  mental  health  treatment  (Roehrle  & 
Strouse,  2008)  and  adaptive  interpersonal  functioning  (Haber,  Co¬ 
hen,  Lucas,  &  Baltes,  2007;  Holt-Lunstad,  Smith,  &  Layton, 
2010).  Telehealth  can  facilitate  social  connections  and  support  that 
may  not  otherwise  be  feasible.  For  example,  telehealth  procedures 
have  been  used  to  promote  relationship  building  via  remote  contact 
with  socially  isolated  caregivers  of  individuals  suffering  with 
dementia  who  often  must  sacrifice  their  own  social  support  to 
provide  care  for  others  (Wright,  Bennet,  &  Gramling,  1998).  In 
this  example,  telehealth  allowed  these  caregivers  to  achieve  mean¬ 
ingful  social  interactions  without  disrupting  their  caregiving  re- 
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sponsibilities.  Additionally,  Blank,  Chang,  Fox,  Lawson,  and 
Modlinski  (1996)  have  reported  that  among  patients  at  risk  for 
treatment  drop-out,  individuals  who  received  a  telephone  call 
between  sessions  were  more  likely  to  attend  their  next  session. 

HBTMH  also  allows  for  more  contact  between  patients  and 
practitioners  in  cases  where  treatment  frequency  is  affected  by 
logistical  factors  such  as  travel  time  and  cost.  Treatments  that 
require  a  significant  amount  of  between  session  homework  (e.g., 
activity  monitoring,  exposure  exercises,  symptom  tracking,  etc.) 
would  benefit  from  the  closer  treatment  intervals  that  HBTMH  can 
provide.  HBTMH  may  also  allow  practitioners  to  more  effectively 
monitor  symptoms,  assess  risk,  and  intervene  before  a  patient 
worsens  or  a  crisis  develops.  For  example,  weekly  contact  via 
HBTMH  with  a  patient  that  is  at  high  risk  for  suicide  and  who  lives 
several  hours  away  from  their  treatment  practitioner’s  office  could 
provide  an  opportunity  to  detect  early  stage  increases  in  suicidal 
ideation  that  may  not  be  possible  with  less  frequent  in-person  (e.g., 
monthly)  contact.  Moreover,  frequent  contact  that  fosters  a  caring 
social  connection  between  practitioner  and  patient  may  in  itself 
help  to  protect  against  suicidal  behavior.  This  idea  is  consistent 
with  “caring  letters,”  the  suicide  prevention  intervention  that  in¬ 
volves  the  maintenance  of  a  caring  connection  through  brief  mes¬ 
sages  of  care  between  treatment  practitioners  and  patients  follow¬ 
ing  treatment  or  hospital  discharge  (Luxton,  June  &  Comtois, 
2013;  Motto,  1976).  Although  the  routine  patient  contact  made 
through  HBTMH  services  is  not  the  same  in  form  or  intent  as  the 
“caring  letters”  intervention,  HBTMH  may  nonetheless  provide 
the  same  sort  of  regular,  empathetic  contact  by  a  care  practitioner 
that  may  be  especially  important  for  high-risk  patients  who  are 
geographically  isolated  or  who  lack  supportive  social  connections. 

Access  to  Contextual  Information 

Another  potential  benefit  of  HBTMH  is  the  ability  to  observe, 
when  clinically  appropriate  and  with  the  consent  of  the  patient, 
information  in  real-time  about  the  patient’s  living  environment. 
The  capability  to  view  the  patient’s  home  environment  with  VTC 
technology  can  provide  a  short-cut  to  valuable  information  for  the 
practitioner.  For  example,  if  a  provider  discovers  that  a  patient’s 
pet  is  present  during  a  session,  that  information  can  be  used  to 
spark  a  discussion  about  how  much  the  pet  means  to  the  patient 
and  what  that  relationship  provides  to  them.  This  also  provides  the 
practitioner  with  information  about  the  patient’s  activity  level  and 
an  opportunity  to  plan  for  meaningful  and  important  caretaking 
behavior.  Alternatively,  when  treating  a  patient  with  insomnia  the 
practitioner  could,  with  permission,  “tour”  the  patient’s  sleeping 
environment.  The  practitioner  could  then  identify  factors  (e.g., 
presence  of  bright  alarm  clocks,  lack  of  window-coverings,  or 
other  environmental  characteristics)  that  could  be  addressed  as  part 
of  treatment  (Henry,  Rosenthal,  Dedrick,  &  Taylor,  2013).  Simi¬ 
larly,  for  a  patient  seeking  treatment  for  substance  abuse  or  de¬ 
pendence,  the  practitioner  and  patient  could  work  collaboratively, 
in  real  time,  to  remove  or  dispose  of  drug  related  paraphernalia  or 
other  relapse  triggers  in  the  patient’s  environment  (Benavides- 
Vaello,  Strode,  &  Sheeran,  2013). 

The  capability  to  view  a  patient’s  environment  and  personal 
effects  in  their  home,  when  clinically  relevant,  may  also  contribute 
to  rapport  and  connectedness  in  a  manner  akin  to  the  joining 
process  that  occurs  when  a  therapist  physically  conducts  treatment 


with  a  family  in  their  own  home,  wherein  the  practitioner  gains 
experiential  insight  into  the  day-to-day  routines,  context,  and  con¬ 
tingencies  operating  in  the  family  system  (Reiter,  2000).  For 
example,  a  patient  may  have  a  dedicated  room  for  a  hobby,  sports 
team,  or  another  personal  interest,  and  those  interests  or  activities 
can  be  brought  into  the  therapeutic  context  to  build  a  stronger  and 
more  personalized  therapeutic  relationship.  However,  practitioners 
should  consider  the  clinical  implications  of  taking  an  active  role  in 
assessing  a  patient’s  home  environment  versus  educating  and 
encouraging  the  patient  to  assess  their  own  environment.  Practi¬ 
tioners  must  also  be  sensitive  to  the  privacy  needs  of  patients  and 
consider  them  based  on  a  given  patient’s  clinical  presentation  and 
needs. 

As  has  been  noted  elsewhere  (Luxton,  Pruitt,  &  Osenbach, 
2014;  Wagnild,  Leenknecht,  &  Zauher,  2006),  some  clinically 
relevant  data  (e.g.,  olfactory  impressions  of  a  client  with  a  history 
of  alcohol  dependence,  view  of  the  client’s  entire  body  with  which 
to  gauge  psychomotor  agitation)  may  go  unobserved  when  using 
telehealth  technologies.  Conducting  treatment  remotely,  whether 
to  a  supervised  clinic  or  the  home,  requires  practitioners  to 
ply  their  craft  in  a  flexible  manner  that  suits  the  context.  This  is 
true  not  only  in  terms  of  clinical  data  gathering  but  also  with 
regard  to  the  adjustments  that  must  be  made  to  how  treatment  is 
conducted.  For  example,  the  practitioner  may  need  to  account  for 
others  that  may  be  in  the  patient’ s  environment,  make  sure  that  the 
on-screen  image  of  the  practitioner  can  be  seen  and  heard  clearly, 
and  discuss  with  the  patient  how  to  manage  potential  disruptions  to 
session  that  occur  in  the  patient’s  home.  It  is  important  to  note  that 
these  potential  problems  are  manageable,  and  that  as  VTC  tech¬ 
nology  continues  to  improve  (e.g.,  easily  accessible  smartphone 
and  tablet  platforms,  the  ability  to  share  materials  between  sites, 
integration  with  the  electronic  medical  record,  etc.)  it  is  likely  that 
the  impact  of  these  potential  disruptions  will  be  reduced  (Brooks, 
Turvey,  &  Augusterfer,  2013). 

Patient  and  Practitioner  Safety 

HBTMH  can  also  help  to  improve  the  safety  of  patients  which 
may  lead  to  improved  treatment  adherence  and  outcomes.  For 
example,  an  elderly  patient  with  reduced  mobility  and  difficulty 
driving  may  be  better  served  with  care  delivered  to  their  home, 
rather  than  having  to  navigate  the  risks  of  attending  an  in-person 
session.  Similarly,  patients  living  in  locations  with  harsh  winter 
environments  may  benefit  from  avoiding  long  trips  on  treacherous 
roads. 

A  primary  safety  concern  associated  with  HBTMH  centers 
around  the  management  of  situations  when  a  patient  is  in  crisis  or 
threatens  harm  to  self  or  others  at  the  end  of  a  session  or  just  before 
the  patient  unexpectedly  ends  the  session.  A  decreased  sense  of 
control,  unfamiliarity  with  safety  procedures,  and  liability  con¬ 
cerns  are  all  factors  that  may  influence  practitioners’  confidence  in 
providing  HBTMH  services  (Luxton,  Sirotin,  &  Mishkind,  2010). 
However,  there  have  not  been  any  published  studies  that  we  are 
aware  of  to  suggest  that  HBTMH  is  less  safe  than  in-office  care. 
The  available  literature  indicates  that  patient  safety  during  TMH, 
including  HBTMH,  can  be  effectively  managed  with  appropriate 
training  and  safety  planning  (Gros  et  al.,  2011;  Luxton,  Sirotin, 
Mishkind,  2010).  Moreover,  recently  published  guidelines  (e.g., 
American  Psychological  Association  [APA],  2013;  American 
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Telemedicine  Association  [ATA],  2013),  a  practice  recommenda¬ 
tions  review  (Luxton,  O’Brien,  McCann,  &  Mishkind,  2012),  and 
a  TMH  guidebook  (Kramer,  Ayers,  Mishkind,  &  Norem,  2011)  all 
discuss  and  provide  guidance  regarding  safety  management  and 
are  available  to  TMH  practitioners. 

Safety  planning  during  HBTMH  may  have  some  clinical  bene¬ 
fits  that  have  not  been  previously  discussed  in  the  literature.  In 
particular,  the  process  of  working  collaboratively  with  patients  to 
establish  a  safety  plan  may  in  itself  have  a  therapeutic  benefit. 
Additionally,  the  identification  of  another  person  at  the  patient’s 
location  who  can  assist  during  emergencies  or  with  technical 
issues  during  HBTMH  sessions  is  a  TMH  practice  recommenda¬ 
tion  (ATA,  2013;  Luxton  et  al.,  2012).  If  the  support  provided  by 
this  person  matches  the  patient’s  needs  in  a  moment  of  crisis,  their 
involvement  is  likely  to  increase  feelings  of  social  connectedness 
(Callaghan  &  Morrissey,  2008),  help  to  establish  therapeutic  alli¬ 
ance  by  showing  concern  for  the  patient’s  well  being,  and  provide 
the  patient  with  a  sense  of  control.  This  idea  is  consistent  with 
clinical  interventions  that  emphasize  collaborative  treatment  plan¬ 
ning  approaches  such  as  Collaborative  Assessment  and  Manage¬ 
ment  of  Suicidality  (CAMS;  Jobes,  2006).  Collaborative  safety 
planning  may  also  provide  the  patient  with  increased  confidence 
that  there  is  a  plan  to  resolve  emergencies  and  therefore  contribute 
to  improved  comfort  in  and  acceptance  of  the  HBTMH  process. 

Home-based  TMH  may  also  provide,  in  some  cases,  safety 
benefits  for  practitioners.  Consider  that  nonfatal,  job-related  vio¬ 
lent  crime  perpetrated  against  mental  health  professionals  by  pa¬ 
tients  occur  at  a  rate  of  68.2  per  1,000,  approximately  four  times 
greater  than  the  rate  for  non-psychiatrist  physicians  and  approxi¬ 
mately  three  times  greater  than  the  rate  for  nurses  (Anderson  & 
West,  2011;  Friedman,  2006).  Thus,  patients  with  a  history  of 
violence  toward  treatment  practitioners  or  who  are  at  an  elevated 
risk  for  violence  may  be  appropriate  candidates  for  remote  services 
because  the  potential  for  physical  harm  to  treatment  staff  is  min¬ 
imized  by  the  physical  distance  between  the  patient  and  practitio¬ 
ner  (Luxton  et  al.,  2012).  In  any  situation,  the  evaluation  of  the 
entire  range  of  safety  risks,  including  risks  for  the  patient,  family 
members,  and  staff  members  (if  considering  in-office  care)  should 
be  considered  when  determining  the  most  appropriate  treatment 
services  option. 

Improved  Privacy  and  Concerns  About  Stigma 
Associated  With  Care  Seeking 

One  of  the  most  significant  benefits  of  HBTMH  is  that  it  may 
help  to  overcome  concerns  about  privacy  and  stigma  associated 
with  seeking  care  in  person  (Olden,  Cukor,  Rizzo,  Rothbaum,  & 
Difede,  2010).  Concerns  about  privacy  may  be  an  issue  when 
clinic  areas  or  waiting  rooms  are  highly  visible  to  others.  Patients 
who  forgo  seeking  treatment  due  to  these  concerns  may  be  willing 
to  participate  in  care  when  it  is  provided  in  the  privacy  of  their  own 
home. 

Although  HBTMH  may  offer  improved  privacy  in  some  areas, 
additional  privacy-related  concerns  may  arise  related  to  the  use  of 
technology  and  the  lack  of  physical  presence  inherent  HBTMH 
services.  In  particular,  concerns  about  the  security  of  data  trans¬ 
mission  (e.g.,  over  the  Internet)  and  data  storage  may  influence 
both  patients’  and  practitioners’  willingness  to  engage  in  TMH 
services.  The  use  of  VTC  in  the  home  also  presents  some  addi¬ 


tional  privacy  concerns  because  it  is  possible  to  view  the  private 
home  environment  of  the  patient,  and  family  members  or  room¬ 
mates  may  walk  in  on  or  overhear  session  content.  Further,  pa¬ 
tients  may  have  concerns  about  whether  there  may  be  someone 
else  in  the  practitioner’s  office  that  may  be  able  to  hear  the 
conversation. 

It  is  therefore  important  that  practitioners  communicate  to  pa¬ 
tients  that  privacy  and  data  security  are  taken  seriously  and  that 
there  are  procedures  in  place  to  maximize  privacy  and  data  secu¬ 
rity.  The  steps  taken  by  the  provider  as  well  as  those  required  of 
the  patient  should  be  addressed  during  the  informed  consent  pro¬ 
cess  before  commencing  treatment  (APA,  2013;  Luxton  et  al., 
2014).  For  example,  discussion  of  data  encryption  and  risks,  rec¬ 
ommendation  for  securing  the  treatment  environment  (e.g.,  closing 
doors  and  windows),  and  negotiating  with  family  members  in  the 
house  about  dedicating  time  and  space  for  treatment  sessions  are 
recommended.  These  steps  may,  in  some  cases,  be  clinically 
beneficial.  For  instance,  these  activities  may  help  to  facilitate 
communication  of  needs  between  a  patient  and  their  partner  or 
provide  a  patient  with  a  sense  of  control  and  support  in  achieving 
their  goals.  It  is  important  to  note  that  HBTMH  may  also  burden 
the  patient  with  assuring  that  they  have  a  private  space  that  can  be 
secured  during  appointment  times.  Care  practitioners  thus  need  to 
be  cognizant  of  this  issue  when  practicing  HBTMH  and  to  work 
with  their  patients  to  achieve  optimal  therapeutic  conditions. 

Conclusions 

HBTMH  is  an  option  to  improve  access  to  mental  health  ser¬ 
vices  by  bringing  care  into  the  convenience  and  comfort  of  a 
patient’s  home.  Advancements  in  both  the  sophistication  and  om¬ 
nipresence  of  Internet  connectivity,  Webcams,  videoconferencing 
software,  and  smart  mobile  devices  have  made  HBTMH  more 
feasible  now  than  ever  before.  A  growing  literature  base  suggests 
that  treatments  provided  via  HBTMH  may  be  as  effective  and 
acceptable  as  in-person  services  with  the  potential  to  offer  some 
unique  and  important  clinical  benefits.  These  benefits  include 
enhanced  social  support,  safety,  understanding  of  patient’s  home 
context,  and  privacy.  HBTMH  is  also  in  line  with  larger  health 
care  goals,  including  field-wide  progress  toward  integrated  care 
and  a  patient-centered  medical  home  model  (PCMH;  Stange  et  al., 
2010).  These  aspects  of  home-based  care  are  the  driving  force 
behind  initial  efforts  to  implement  large  scale  HBTMH  programs 
in  both  public  (e.g.,  VA)  and  private  health  care  systems.  HBTMH 
is  a  critical  component  of  integrated  care  because  it  enables  pri¬ 
mary  care  and  mental  health  services  to  be  efficiently  and  conve¬ 
niently  integrated  without  requiring  patients  to  attend  two  or  more 
clinics  for  services.  Moreover,  because  telehealth  allows  for 
shorter  follow-up  or  assessment  sessions,  integrated  HBTMH  care 
can  provide  an  economic  benefit  to  both  patients  and  health  care 
systems. 

There  is  still  much  work  that  remains  to  be  done  before 
HBTMH  sees  wide-spread  implementation.  As  noted  by  Brooks, 
Turvey,  and  Augusterfer  (2013),  three  of  the  biggest  issues  facing 
the  practice  of  TMH,  and  by  extension  HBTMH,  are  (a)  the 
provision  of  clinical  services  via  this  modality  may  not  be  eligible 
for  medical  reimbursement;  (b)  it  is  unknown  how  remote  services 
affect  and  are  affected  by  laws  regulating  the  practice  of  mental 
health  professionals  across  state  lines  (e.g.,  issues  associated  with 
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mandatory  reporting,  duty  to  warn,  etc.);  and  (c)  changes  need  to 
be  made  to  liability  standards  to  make  this  mode  of  treatment 
delivery  a  feasible  practice.  Although  further  discussion  of  these 
specific  topics  is  beyond  the  scope  of  this  article,  the  issues 
associated  with  liability  highlight  the  challenges  facing  the  adop¬ 
tion  of  new  health  care  models  into  standard  practices.  It  is 
essential  to  support  and  increase  the  access  practitioners  have  to 
HBTMH  training  given  the  liability,  safety,  and  practical  nuances 
of  HBTMH.  We  recommend  that  practitioners  become  familiar 
with  published  practice  guidelines  by  professional  organizations 
such  as  the  American  Psychological  Association  (APA,  2013)  and 
the  American  Telemedicine  Association  (ATA,  2013).  We  also 
recommend  that  practitioners  seek  training  that  is  available 
through  several  organizations  such  as  the  American  Psychological 
Association  and  the  American  Telemedicine  Association. 

Additional  research  is  also  needed  to  move  HBTMH  practice 
forward.  Limitations  of  the  existing  clinical  effectiveness  literature 
center  around  concerns  about  the  methodology  used,  including 
small  samples,  statistically  underpowered  comparisons,  and  lack 
of  random  assignment  to  treatment  conditions  (Hyler,  Gangure,  & 
Batchelder,  2005;  Richardson  et  al.,  2009).  Other  limitations  that 
have  been  noted  include  a  lack  of  diverse  study  samples  (i.e., 
clinical  presentations,  socioeconomic  diversity,  various  cultural 
aspects),  failure  to  compare  HBTMH  with  face-to-face  compara¬ 
tors  using  gold-standard  treatments,  and  failure  to  use  conserva¬ 
tive,  gold-standard  measures  of  clinical  outcomes,  and  treatment 
compliance  (Aoki,  Dunn,  Johnson-Throop,  &  Turley,  2003;  Hilty 
et  al.,  2004;  Roine,  Ohinmaa,  &  Hailey,  2001).  To  address  these 
issues,  the  field  must  move  away  from  small  studies  that  are 
underpowered  to  detect  between  group  differences,  fail  to  ran¬ 
domly  assign  participants  to  treatment  groups,  or  include  homog¬ 
enous  samples  that  hamper  generalization  of  findings.  More  rig¬ 
orous,  randomized  controlled  trials  with  adequately  large  and 
diverse  samples  are  needed  in  order  to  provide  meaningful  con¬ 
clusions  that  apply  to  broad  populations  of  potential  HBTMH 
consumers.  Further,  HBTMH  should  be  investigated  using  treat¬ 
ments  that  have  been  empirically  validated  and  make  use  of 
gold-standard  outcome  measures. 

Just  as  it  is  important  to  understand  who  may  benefit  from 
HBTMH,  understanding  who  is  less  likely  to  benefit  from  it  is 
equally  important — a  concept  that  harkens  back  to  the  guiding 
question  posed  by  Paul  (1967):  “What  treatment,  by  whom,  is  most 
effective  for  this  individual,  with  that  specific  problem,  and  under 
which  set  of  circumstances?”  (p.  111).  In  the  case  of  HBTMH, 
practitioners  must  remain  cognizant  of  whether  the  modality  is  the 
best  option  for  any  given  patient  based  on  that  patient’s  needs. 
Particular  attention  must  also  be  paid  to  both  patient  and  practi¬ 
tioner  preferences  for — and  acceptance  of — HBTMH  in  order  to 
make  this  modality  a  viable  treatment  option  with  maximum 
benefit. 

In  conclusion,  HBTMH  services  are  highly  relevant  because 
they  have  the  potential  to  help  meet  the  current  and  future  demands 
on  the  national  (U.S.)  health  care  system.  The  implementation  of 
HBTMH  may  yield  improved  mental  health  care  access  to  the 
nearly  80  million  Americans  who  reside  in  areas  without  a  suffi¬ 
cient  number  of  mental  health  care  practitioners  to  meet  the  needs 
of  those  communities  (U.S.  Health  and  Human  Services  Health 
Resources  and  Services  Administration,  2013).  Even  in  urban 
environments,  where  mental  health  professionals  are  in  greater 


numbers,  cost,  transportation,  and  time  constraints  often  prevent 
people  from  seeking  mental  health  services  (Novotney,  2011). 
Telehealth  may  function  as  a  balancing  factor  in  the  misdistribu- 
tion  of  mental  health  practitioners  by  providing  clinical  care  at  a 
distance  in  both  rural  and  urban  settings.  Barriers  that  currently 
prevent  individuals  from  seeking  treatment  may  be  greatly  reduced 
by  the  use  of  HBTMH  services  and  may  offer  additional  clinical 
benefits  that  improve  treatment  outcomes,  health  promotion,  and 
overall  well-being. 
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